Dstack is a minimalist testnet for self-replicating Confidential VMs (CVMs). The orchestration contract lives on Sepolia. The sample application serves an unstoppable HTTPs website. Browse to https://dstack-mockup.ln.soc1024.com for a demo greeting:
Welcome to Dstack with unstoppable TLS!
The total repo size comes in under 1000 lines of code, mainly bash scripts to build and run the VM and python modules to manage the replication and serve the app.
Dstack provides a flexible environment for apps running on the guest VM. Your app is a plain Docker container archive (see the greeting app at ./app-example/). Apps can access:
- Encrypted persistent volume:
/mnt/encrypted_data/ - Untrusted host volume:
/mnt/host_volume/ - Per-app hardened keys:
http://dstack-guest/getkey/<tag> - EVM-friendly remote attestation:
http://dstack-guest/attest/<tag>/<appdata>
To play along with the Dstack test network, you need a linux environment with qemu and kvm, Sepolia testnet coins from a faucet, and a Sepolia API key. You do NOT need a trusted hardware capable machine like SGX/TDX.
- Building the VM and sample app: under 5 minutes
- Booting and joining the network: under 60 seconds
- Disk space required: ~4GB
When the Dstack VM starts up, it goes straight to the Replicatoor to get a copy of the network-wide shared secret. This works by posting a transaction on-chain, passing along a remote attestation. Any existing node on the network can verify the remote attestation, and respond on-chain. See a sample Register transaction and an Onboard transaction.
The protocol is illustrated below:
To keep gas costs down, the raw remote attestation quotes are only handled off-chain. A bootstrapping quote from the first enclave is checked into ./auditing so it can be inspected at auditing time. Otherwise, quotes from a new node are inspected by an existing node before handing over a copy of the key.
Note that the implementation here works with actual TDX DCAP quotes. But if you run the VM without a TDX, then attestations are provided from a dummy service.
Dstack uses the replicated secret to derive a shared TLS private key for the HTTPS server. To proactively check a certificate (similar to RATLS or aTLS), verify a signature from the EVM-friendly remote attestation over the public key in the certificate.
Nodes generate a Certificate Signing Request (CSR) after obtaining the key. We get the certificate issued by letsencrypt, and it shows up on certificate transparency websites like https://crt.sh/?q=dstack-mockup.ln.soc1024.com. To rely on certificate transparency and public auditors, listed at https://crt.sh/?q=dstack-mockup.ln.soc1024.com.
In Dstack, a smart contract is the owner of the cluster. The smart contract keeps track of the desired container image (see the field in the explorer), and the Kubernethes script monitors for changes and reloads the container as needed.
It's not necessary to have TDX to play along, since this image is meant to be easy to run in a environment. Instead it's just necessary to install qemu and libvirt.
apt-get install qemu qemu-kvm libvirt-daemon-system libvirt-clients bridge-utils virt-managerLibvirt will read from the system kernel, so I go ahead and allow read access:
sudo chmod a+r /boot/vmlinuz*
We have to start by downloading a base image if you don't have it. The ubuntu minimal is under 300MB, and after we're done installing packages it will be under 2GB.
wget https://cloud-images.ubuntu.com/minimal/releases/noble/release-20240903/ubuntu-24.04-minimal-cloudimg-amd64.img
ln -s $PWD/ubuntu-24.04-minimal-cloudimg-amd64.img ./ubuntu-minimal.img
chmod -w ubuntu-24.04-minimal-cloudimg-amd64.imgTo kick off the build process, run
./build_vm.shIt's convenient when developing to take a copy of the image after apt get update. For this you can have a look at ./build_vm_dev.sh. It's necessary to rm ubuntu_vm.step1.img to make it rebuild the intermediate snapshot.
To build the sample application, just use Docker and store it in the untrusted host volume so the podman in the guest VM can access it.
./build_app.shor
docker build -t app-example:latest app-example
docker save -o ./host_volume/app-example.tar app-example:latestTODO:
- we could store our own repo mirror of the packages we installed, along with their signatures from package maintainers or evidence of how widely they were distributed
- we could use virt-diff to enumerate the differences in images
The main goal will be to take our VM and connect it to the testnet network, retrieving a copy of the shared key.
Once we have the shared key, the VM will use this key to mount an encrypted disk image and run the app.py.
First we need to configure host.env.
The host services are not part of the enclave, but are responsible for signing the actual transactions and paying gas.
See host.env.example:
PRIVKEY=
ETH_API_KEY=We are looking forward to moving to on-chain PCCS. But for now we still have to do a workaround to deal with platform collateral caching, which requires us to run another tool that interacts with Intel's service.
git submodule update --init --recursive
cd dummy-tdx-dcap
make build-httpserverThe last setup to run the host service is this:
sudo apt-get install tmux dumpasn1
pip install -r requirements.txtWe have to fetch the current certificate from crt.sh and store it in the untrusted host volume.
python scripts/getcert.pyFinally here's a little tmux script that runs the host services and the VM
./tmuxdemo.shHere's a summary of the components:
The provided host_service.py script monitors the Sepolia blockchain, looking for valid requests, and if they are valid then passes them to the enclave. The resulting ciphertext is then posted on-chain, where the node attempting to register will be able to see it.
Since the blobs are too expensive to send entire quotes, we're just running a service, see host.env.example and pubsub.py.
Ideally we can provide a few alternatives here. The enclave doesn't really care how the host provides it.