8000 GitHub - cyril-k/vllm-tool-calling: Examples for `tool_call` with vLLM
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

cyril-k/vllm-tool-calling

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

1 Commit
 
 
 
 
 
 
 
 
 
 

Repository files navigation

tool_call examples with vLLM

Start with installing poetry and launching the shell:

poetry install --no-root
poetry shell

Don't forget to add your HF_TOKEN to env, somemodels require you to accept terms of use or demand permission to download weights:

export HF_TOKEN=provide-your-own

You may execute scripts in vllm-tool-calling.ipynb. To launch vLLM server, execute corresponding commands in poetry shell.

About

Examples for `tool_call` with vLLM

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published
0