| 序号 | Tag | 构建时间 | 指令集 | PyTorch版本 | New | WEB | 架构 |
|---|---|---|---|---|---|---|---|
| 01 | v2025.04.12-action | 2025.4.12 | NATIVE | 2.6.0+cu126 | 最新镜像 | ❌ | amd64 |
| 02 | v2025.04.13-action-web | 2025.4.13 | NATIVE | 2.6.0+cu126 | 最新镜像 | ✅ | amd64 |
PyTorch基础镜像：https://hub.docker.com/r/pytorch/pytorch/tags
services:
ktransformers:
image: registry.cn-hangzhou.aliyuncs.com/joybo/ktransformers:v2025.04.12-action
container_name: ktransformers
hostname: ktransformers
environment:
- TZ=Asia/Shanghai
# - NVIDIA_VISIBLE_DEVIC
8000
ES=0
volumes:
- ./DeepSeek-V2-Lite-Chat-GGUF:/app/model/DeepSeek-V2-Lite-Chat-GGUF
ports:
- "10002:10002"
runtime: nvidia
deploy:
resources:
reservations:
devices:
- driver: nvidia
# device_ids: ['0']
capabilities: [gpu]
stdin_open: true
tty: true
restart: no
networks:
- ktransformers
entrypoint: ["python", "-m", "ktransformers.local_chat", "--model_path=/app/model/DeepSeek-V2-Lite-Chat", "--gguf_path=/app/model/DeepSeek-V2-Lite-Chat-GGUF"]
networks:
ktransformers:
driver: bridge
name: ktransformers🌐 FastAPI：http://<IP地址>:10002/docs，具体请参见 fastapi 官方文档
services:
ktransformers:
image: registry.cn-hangzhou.aliyuncs.com/joybo/ktransformers:v2025.04.13-action-web
container_name: ktransformers
hostname: ktransformers
environment:
- TZ=Asia/Shanghai
# - NVIDIA_VISIBLE_DEVICES=0
volumes:
- ./DeepSeek-V2-Lite-Chat-GGUF:/app/model/DeepSeek-V2-Lite-Chat-GGUF
ports:
- "10002:10002"
runtime: nvidia
deploy:
resources:
reservations:
devices:
- driver: nvidia
# device_ids: ['0']
capabilities: [gpu]
stdin_open: true
tty: true
restart: no
networks:
- ktransformers
entrypoint: ["ktransformers", "--model_path=/app/model/DeepSeek-V2-Lite-Chat", "--gguf_path=/app/model/DeepSeek-V2-Lite-Chat-GGUF", "--port=10002", "--web=True"]
networks:
ktransformers:
driver: bridge
name: ktransformers由于官方的WEB的问题，无法正常使用WEBUI聊天，需要先修改config.js (/workspace/ktransformers/ktransformers/website/public/config.js)
window.configWeb = {
apiUrl: '/v1',
port: 8080,
};【预下载的模型配置】
1、DeepSeek-R1：/app/model/DeepSeek-R1
2、DeepSeek-V3-0324：/app/model/DeepSeek-V3-0324
3、DeepSeek-V2-Lite-Chat：/app/model/DeepSeek-V2-Lite-Chat
【启动命令：LocalChat】（GGUF文件需要下载到对应的目录内，仅允许存在GGUF文件，且不允许存在多个不同量化版本的GGUF）
1、DeepSeek-R1：python -m ktransformers.local_chat --model_path=/app/model/DeepSeek-R1 --gguf_path=/app/model/DeepSeek-R1-GGUF
2、DeepSeek-V3-0324：python -m ktransformers.local_chat --model_path=/app/model/DeepSeek-V3-0324 --gguf_path=/app/model/DeepSeek-V3-0324-GGUF
3、DeepSeek-V2-Lite-Chat：python -m ktransformers.local_chat --model_path=/app/model/DeepSeek-V2-Lite-Chat --gguf_path=/app/model/DeepSeek-V2-Lite-Chat-GGUF
【启动命令：WEB服务】（GGUF文件需要下载到对应的目录内，仅允许存在GGUF文件，且不允许存在多个不同量化版本的GGUF）
1、DeepSeek-R1：ktransformers --model_path=/app/model/DeepSeek-R1 --gguf_path=/app/model/DeepSeek-R1-GGUF --port=10002 --web=True
2、DeepSeek-V3-0324：ktransformers --model_path=/app/model/DeepSeek-V3-0324 --gguf_path=/app/model/DeepSeek-V3-0324-GGUF --port=10002 --web=True
3、DeepSeek-V2-Lite-Chat：ktransformers --model_path=/app/model/DeepSeek-V2-Lite-Chat --gguf_path=/app/model/DeepSeek-V2-Lite-Chat-GGUF --port=10002 --web=True
【部分GGUF的地址：国内可访问】
1、DeepSeek-R1：https://hf-mirror.com/unsloth/DeepSeek-R1-GGUF
2、DeepSeek-V3-0324：https://hf-mirror.com/unsloth/DeepSeek-V3-0324-GGUF
3、DeepSeek-V2-Lite-Chat：https://hf-mirror.com/mradermacher/DeepSeek-V2-Lite-Chat-GGUF
https://github.com/kvcache-ai/ktransformers.git
https://kvcache-ai.github.io/ktransformers/en/install.html
https://kvcache-ai.github.io/ktransformers/en/api/server/website.html
https://github.com/kvcache-ai/ktransformers/blob/main/doc/en/Docker.md
https://github.com/kvcache-ai/ktransformers/blob/main/Dockerfile