Home ⟣ Setting up Intel SGX

Setting up Intel SGX

Published on September 29, 2023

Setting up Intel SGX

Since my research partially focuses on Intel SGX, I often find myself having to go through the process of setting up Intel SGX on various machines. Unfortunately, this process consists of multiple steps and has changed quite a bit over the years. In addition, most documentation focuses on specific Linux distributions, usually older versions, such as Ubuntu 20.04 LTS, and many resources handwave the process by providing a Docker image instead. In the meanwhile the Intel SGX driver has finally been mainlined into the Linux kernel [1], and as such the linux-sgx-driver is no longer necessary. As a result, my notes focus on setting up Intel SGX on a system with a Linux kernel that ships the Intel SGX driver (Linux 5.11 and later).

Furthermore, there are currently two different implementations of Intel SGX: EPID (Enhanced Privacy ID) and DCAP (Data Center Attestation Primitives). I won't go into the details of the differences between these two implementations, but you can read more about these in the EPID whitepaper and the DCAP whitepaper. While developers should aim to use DCAP-based Intel SGX rather than EPID-based Intel SGX, as the latter is slowly being phased out (the Intel Xeon E-23xx series is the latest platform to support EPID-based Intel SGX), my notes will focus on setting up Intel SGX on EPID-based platforms as that is what I have mostly been working with. However, I might update these notes in the future for Intel DCAP.

Finding a compatible platform

First, we will need to find a platform that offers Intel SGX. This depends on both the processor and the motherboard, and more specifically the UEFI implementation provided by the OEM.

To determine if a given processor supports Intel SGX or not, you can look up the specification of the processor on https://ark.intel.com. If Intel SGX is supported by the processor, there will be an entry for Intel Software Guard Extensions (Intel SGX) with either Yes with Intel ME or Yes with Intel SPS.

Here is a list of processor families with some processors in each family supporting the EPID-based version of Intel SGX:

Here is a similar list for processors supporting the DCAP-based Intel SGX:

Determining whether the motherboard itself supports Intel SGX beforehand is a little bit more difficult, but it usually helps to check the motherboard manual. Usually the manual will have an overview of all the options provided by the BIOS/UEFI, including an option for Software Guard eXtensions (SGX) which can be set to Disabled, Software-Controlled or Enabled. If such an option is not available, I recommend looking for a different motherboard/platform instead.

Depending on the platform, it may not be possible to set the Software Guard eXtensions (SGX) setting to Enabled. Instead some platforms only allow you to configure the setting to Software-Controlled or Disabled. This is perfectly fine, as we can still enable Intel SGX from the operating system when the setting is set to Software-Controlled.

Enabling Intel SGX

Enter the UEFI setup menu to enable Intel SGX. A quick way of entering the UEFI setup menu on Ubuntu, is by invoking the following command from a terminal:

sudo systemctl reboot --firmware-setup

Alternatively, you can enter the UEFI setup menu by pressing a specific key immediately after the machine is powered on (usually F2 or Del).

In my case the option to enable Intel SGX can be accessed by selecting the Advanced tab, followed by the CPU Configuration menu, where it is available as Software Guard Extensions (SGX). However, the option may be available elsewhere in your UEFI setup menu depending on the exact platform. Make sure that the option is set to either Enabled or Software-Controlled as shown in Figure 1.

SGX UEFI setting

Figure 1: The Software Guard Extensions (SGX) setting in the UEFI setup menu.

On some platforms, there may also be a setting named SGX Launch Control Policy with the options Intel Locked, Locked and Unlocked. Ensure that this setting is set to Unlocked, otherwise the SGX kernel driver will fail to load.

Software-Controlled SGX

If the option is set to Software-Controlled, then you will need to clone, build and run the sgx-software-enable tool to enable Intel SGX.

Make sure to install the required build dependencies:

sudo apt -y install build-essential git

Clone the repository as follows:

git clone https://github.com/intel/sgx-software-enable

Then cd into the directory:

cd sgx-software-enable

Run make to build the tool:

make

Run the following command to enable Intel SGX:

sudo ./sgx-enable

Finally reboot the system for the setting to take effect:

sudo reboot

At this point you should have a platform with support for Intel SGX and where the feature has been enabled.

Checking your Intel SGX Setup

Since setting up Intel SGX consists of multiple steps, it is useful if we can check this using a diagnostics tool. Fortunately, Fortanix provides such a tool called sgx-detect. However, to install the tool, we will first need to install Rust.

First, make sure to install curl (not the snap version):

sudo apt -y install curl

Then we can install rustup to manage our Rust toolchain(s):

curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh

Source the newly added environment variables:

source "$HOME/.cargo/env"

To build the sgx-detect tool, we will need to install the nightly version of the Rust toolchain:

rustup install nightly

In addition, we will also need to add the x86_64-fortanix-unknown-sgx target:

rustup target add x86_64-fortanix-unknown-sgx --toolchain nightly

We will also need to install some build dependencies to the build the sgx-detect tool:

sudo apt -y install build-essential pkg-config libssl-dev protobuf-compiler

Finally, we can use Cargo to install the Fortanix tools, including sgx-detect:

cargo install fortanix-sgx-tools sgxs-tools

To run the tool with the appropriate runner, we will need to add the following to .cargo/config:

[target.x86_64-fortanix-unknown-sgx]
runner = "ftxsgx-runner-cargo"

Then we should be able to run sgx-detect, which should output something like the following:

✔  SGX instruction set
  ✔  CPU support
  ✔  CPU configuration
  ✔  Enclave attributes
  ✔  Enclave Page Cache
  SGX features
    ✔  SGX2  ✔  EXINFO  ✔  ENCLV  ✔  OVERSUB  ✔  KSS
    Total EPC size: 188.0MiB
？ Flexible launch control
  ✔  CPU support
  ？ CPU configuration
  ？ Able to launch production mode enclave
✘  SGX system software
  ✘  SGX kernel device
  ✘  libsgx_enclave_common
  ✘  AESM service

🕮  Flexible launch control > CPU configuration
Your hardware supports Flexible Launch Control, but whether it's enabled could not be determined. More information might be available by re-running this program with sudo. Would you like to do that?
(not supported yet)

(run with `--verbose` for more details)

More information: https://edp.fortanix.com/docs/installation/help/#flc-cpu-configuration

🕮  SGX system software > SGX kernel device
Permission denied while opening the SGX device (/dev/sgx/enclave, /dev/sgx or /dev/isgx). Make sure you have the necessary permissions to create SGX enclaves. If you are running in a container, make sure the device permissions are correctly set on the container.

(run with `--verbose` for more details)

More information: https://edp.fortanix.com/docs/installation/help/#sgx-driver

🕮  SGX system software > AESM service
AESM could not be contacted. AESM is needed for launching enclaves and generating attestations.

Please check your AESM installation.

(run with `--verbose` for more details)

More information: https://edp.fortanix.com/docs/installation/help/#aesm-service

As we can see, we are still missing parts of the software stack, such as the AESM service and the driver.

Intel SGX SDK

Up next, we will build and install the Intel SGX SDK for Linux.

First, we will have to install a number of build dependencies to build the Intel SGX SDK for Linux:

sudo apt -y install build-essential ocaml ocamlbuild automake autoconf libtool wget python-is-python3 libssl-dev git cmake perl libcurl4-openssl-dev protobuf-compiler libprotobuf-dev debhelper cmake reprepro unzip pkgconf libboost-dev libboost-system-dev libboost-thread-dev lsb-release libsystemd0

To build the Intel SGX SDK for Linux, I recommend using one of the reproducible tags of the linux-sgx repository, as these are used to build the Docker images. As of writing, the latest version is sgx_2.21_reproducible. We can clone that tag as follows:

git clone https://github.com/intel/linux-sgx.git -b sgx_2.21_reproducible

Then cd into the repository:

cd linux-sgx

To download the submodules and the prebuilt binaries, we need to invoke make preparation as follows:

make preparation

Then we can build the installer for the Intel SGX SDK for Linux as follows:

make sdk_install_pkg

If the installer built successfully, you should be able to run it as follows:

./linux/installer/bin/sgx_linux_x64_sdk_2.21.100.1.bin

It will prompt you whether you want to install it in the current directory:

Do you want to install in current directory? [yes/no] :

Type no, followed by Enter. Instead type your home directory (e.g. /home/user), such that it will be installed as /home/user/sgxsdk. It should output something like the following:

Unpacking Intel SGX SDK ... done.
Verifying the integrity of the install package ... done.
Installing Intel SGX SDK ... done.
/tmp/sgx-sdk-72hs80 ~
~
uninstall.sh script generated in /home/user/sgxsdk

Installation is successful! The SDK package can be found in /home/user/sgxsdk

Please set the environment variables with below command:

source /home/user/sgxsdk/environment

Run the following to source the newly created environment variables:

source ~/sgxsdk/environment

Finally, add the following to the bottom of your ~/.bashrc:

. "$HOME/sgxsdk/environment"

Intel SGX PSW

Now that we have the Intel SGX SDK installed, we can build the Intel SGX PSW which contains the platform software, i.e. the AESM service that needs to be installed. First, make sure your terminal is pointing to the linux-sgx repository:

cd linux-sgx

Then we can simply build the PSW installer package as follows:

make psw_install_pkg

If everything went well, you should be able to run the installer as root:

sudo ./linux/installer/bin/sgx_linux_x64_psw_2.21.100.1.bin

After installing the Intel SGX PSW package, we should have the AESM service up and running. Invoking the sgx-detect command should now tell us that we have a SGX kernel device, libsgx_enclave_common as well as the AESM service:

Detecting SGX, this may take a minute...
✔  SGX instruction set
  ✔  CPU support
  ✔  CPU configuration
  ✔  Enclave attributes
  ✔  Enclave Page Cache
  SGX features
    ✔  SGX2  ✔  EXINFO  ✔  ENCLV  ✔  OVERSUB  ✔  KSS
    Total EPC size: 188.0MiB
✔  Flexible launch control
  ✔  CPU support
  ？ CPU configuration
  ✔  Able to launch production mode enclave
✘  SGX system software
  ✔  SGX kernel device (/dev/sgx_enclave)
  ✔  libsgx_enclave_common
  ✔  AESM service
  ✘  Able to launch enclaves
    ✘  Debug mode
    ✔  Production mode

🕮  SGX system software > Able to launch enclaves > Debug mode
The enclave could not be launched.

(run with `--verbose` for more details)

More information: https://edp.fortanix.com/docs/installation/help/#run-enclave-debug

If you see the above output where you may not be able to launch (debug) enclaves, it may help to reboot your system:

sudo reboot

Invoking the sgx-detect command should output the following:

Detecting SGX, this may take a minute...
✔  SGX instruction set
  ✔  CPU support
  ✔  CPU configuration
  ✔  Enclave attributes
  ✔  Enclave Page Cache
  SGX features
    ✔  SGX2  ✔  EXINFO  ✔  ENCLV  ✔  OVERSUB  ✔  KSS
    Total EPC size: 188.0MiB
✔  Flexible launch control
  ✔  CPU support
  ？ CPU configuration
  ✔  Able to launch production mode enclave
✔  SGX system software
  ✔  SGX kernel device (/dev/sgx_enclave)
  ✔  libsgx_enclave_common
  ✔  AESM service
  ✔  Able to launch enclaves
    ✔  Debug mode
    ✔  Production mode
    ✔  Production mode (Intel whitelisted)

You're all set to start running SGX programs!

At this point you should be able to build and run any of the examples in ~/sgxsdk/SampleCode.

References

  1. https://www.phoronix.com/news/Intel-SGX-Linux-5.11

[ #intel #sgx #linux ]

If you like my work or if my work has been useful to you in any way, then feel free to donate me a cup of coffee. Any donation is much appreciated!