dstack simplifies training, fine-tuning, and deploying
generative AI models on any cloud.
Supported providers: AWS, GCP, Azure, Lambda, TensorDock, and Vast.ai.
- [2023/10] Use world's cheapest GPUs with TensorDock (Release)
- [2023/10] Fine-tuning API (Release)
- [2023/09] RAG with Llama Index and Weaviate (Example)
- [2023/09] Deploying LLMs using Python API (Example)
- [2023/08] Fine-tuning Llama 2 using QLoRA (Example)
- [2023/08] Deploying Stable Diffusion using FastAPI (Example)
- [2023/07] Deploying LLMs using TGI (Example)
- [2023/07] Deploying LLMs using vLLM (Example)
Before using dstack through CLI or API, set up a dstack server.
The easiest way to install the server, is via pip:
$ pip install "dstack[all]" -UAnother way to install the server is through Docker.
If you have default AWS, GCP, or Azure credentials on your machine, the dstack server will pick them up automatically.
Otherwise, you need to manually specify the cloud credentials in ~/.dstack/server/config.yml.
For further details, refer to server configuration.
To start the server, use the dstack server command:
$ dstack server
Applying configuration...
---> 100%
The server is running at http://127.0.0.1:3000/.
The admin token is bbae0f28-d3dd-4820-bf61-8f4bb40815daFor additional information and examples, see the following links: