Run SGX in HW mode in k8s #612
Conversation
| RUN cmake \ | ||
| -GNinja \ | ||
| -DCMAKE_CXX_COMPILER=/usr/bin/clang++-13 \ | ||
| -DCMAKE_C_COMPILER=/usr/bin/clang-13 \ | ||
| -DCMAKE_BUILD_TYPE=Release \ | ||
| -DFAASM_SGX_MODE=Simulation \ | ||
| -DFAASM_SGX_MODE=${FAASM_SGX_MODE:-Simulation} \ |
There was a problem hiding this comment.
If FAASM_SGX_MODE is not set, use Simulation, if it is set, use ${FAASM_SGX_MODE}. Link to bash wiki..
There was a problem hiding this comment.
Usually don't need to link to syntax in a PR unless it's some obscure library that you're introducing.
There was a problem hiding this comment.
I always get confused with bash variable's expansion, so added the link and the explanation more for me than for you 🤣
deploy/k8s/upload-sgx.yml
Outdated
| value: "info" | ||
| - name: LD_LIBRARY_PATH | ||
| value: "/build/faasm/third-party/lib:/usr/local/lib" | ||
| - name: WASM_VM |
There was a problem hiding this comment.
This file is a copy of deploy/k8s/upload.yml with just one extra environment variable.
There was a problem hiding this comment.
Rather than mixing these in the same directory I feel like having a totally separate deploy/k8s-sgx directory would be cleaner, then we avoid all the filtering logic in Python. Anything that's shared between vanilla and SGX would go into another new directory, deploy/k8s-common.
There was a problem hiding this comment.
I agree this is cleaner. We now have:
8000
deploy/{k8s-common,k8s-sgx,k8s}.
| @@ -92,6 +92,34 @@ jobs: | |||
| context: . | |||
| tags: faasm/base:${{ env.TAG_VERSION }} | |||
|
|
|||
| 8000 | build-base-sgx: | ||
There was a problem hiding this comment.
This file is getting quite long and verbose, we could redo this with a matrix build like we do the tests. Doesn't need to be in this PR but will add to the todo list.
deploy/k8s/upload-sgx.yml
Outdated
| value: "info" | ||
| - name: LD_LIBRARY_PATH | ||
| value: "/build/faasm/third-party/lib:/usr/local/lib" | ||
| - name: WASM_VM |
There was a problem hiding this comment.
Rather than mixing these in the same directory I feel like having a totally separate deploy/k8s-sgx directory would be cleaner, then we avoid all the filtering logic in Python. Anything that's shared between vanilla and SGX would go into another new directory, deploy/k8s-common.
| RUN cmake \ | ||
| -GNinja \ | ||
| -DCMAKE_CXX_COMPILER=/usr/bin/clang++-13 \ | ||
| -DCMAKE_C_COMPILER=/usr/bin/clang-13 \ | ||
| -DCMAKE_BUILD_TYPE=Release \ | ||
| -DFAASM_SGX_MODE=Simulation \ | ||
| -DFAASM_SGX_MODE=${FAASM_SGX_MODE:-Simulation} \ |
There was a problem hiding this comment.
Usually don't need to link to syntax in a PR unless it's some obscure library that you're introducing.
deploy/k8s-sgx/upload.yml
Outdated
| ports: | ||
| - port: 8002 | ||
| selector: | ||
| role: upload |
There was a problem hiding this comment.
@csegarragonz I think this is now a duplicate of what's in upload-lb.yml. Missed it in my review.
There was a problem hiding this comment.
good catch, thanks. i copied the file before removing this 😓
In this PR I Introduce the necessary changes to allow running SGX functions in hardware mode in a kubernetes cluster. Most importantly, we need to build different image tags for workers with a hardware-mode SGX build.
In addition, when deploying the KNative services we need to: use the new worker docker image, and set the environment variable
WASM_VM=sgxfor both the upload and worker pods.For the upload service, we can either use templated deployment files/env. var substitution or have separate deployment files. For the worker service I think we need a different file, as we have to add sgx-specific reources and we may have to add further options in the worker deployment file for attestation.
I have manually checked that with the current setup, and deploying as follows, we can run SGX functions in HW mode in AKS:
I also bump the code version to test the new "Release" workflow file.