A streamlined command-line tool for interacting with LLM APIs that handles multiple input sources and streaming responses. Supports a wide range of models including Anthropic Claude, OpenAI, and many others.
- Create config directory and copy a model config:
cd ~
git clone https://github.com/LoqLab/yllm.git
cd yllm
mkdir -p ~/.yllm
cp models/claude-3-6-sonnet ~/.yllm/claude-3-6-sonnet
# Add "PATH=$PATH:~/yllm" to your "~/.bashrc" or "~/.zshrc" file- Add your API key:
nano ~/.yllm/claude-3-6-sonnet- Set as default model:
yllm --set-default claude-3-6-sonnetSimple queries:
yllm "What is the capital of France?"
yllm -m mixtral-8x7b "Explain quantum computing" # Specify model# Single file
yllm -f code.py "Explain this code"
# Multiple files
yllm -f doc1.txt -f doc2.txt "Compare these documents"
yllm -f paper.pdf "Summarize this" -f notes.txt "incorporating these points"# Single URL
yllm -u https://example.com "Summarize this page"
# Multiple URLs
yllm -u https://site1.com -u https://site2.com "Compare these websites"# Combine files and URLs
yllm -f research.pdf "Analyze this paper" -u https://related.com "in context of this article"
# Complex combinations
yllm "First, consider this code:" -f code.py \
"Now look at this documentation:" -u https://docs.example.com \
"Explain how they relate"# Implicit stdin (when no other inputs)
echo "Process this" | yllm "What does this mean?"
# Explicit stdin placement
cat data.txt | yllm "Before stdin" -c "after stdin"Save responses:
# Auto-named files in current directory
yllm -z "Query with saved output"
# Specify output directory
yllm -Z /path/to/outputs/ "Query with saved output"
# Convert to PDF
yllm -P "Generate a report" -f data.txtYLLM includes configurations for a wide range of models in the models directory:
- Claude 3.5 Sonnet 2024/06/20 (
models/claude-3-5-sonnet) - Claude 3.5 Sonnet 2024/10/22 (
models/claude-3-6-sonnet) - Claude 3.5 Haiku 2024/10/22 (
models/claude-3-6-haiku)
- Cerebras LLaMA 3.1 Series
- 70B (
models/cerebras-llama3.1-70b) - 8B (
models/cerebras-llama3.1-8b)
- 70B (
- LLaMA Series
- LLaMA 3 405B (
models/llama3-405b) - LLaMA 3 70B (
models/llama3-70b) - LLaMA 70B (
models/llama-70b) - LLaMA 70B Code (
models/llama-70b-code)
- LLaMA 3 405B (
- Mixtral 8x22B (
models/mixtral-8x22b) - Mixtral 8x7B (
models/mixtral-8x7b) - Nous Mixtral (
models/nous-mixtral)
- Groq Gemma 2 9B (
models/groq-gemma2-9b) - Groq LLaMA 3 70B (
models/groq-llama3-70b) - Groq Mixtral 8x7B (
models/groq-mixtral-8x7b)
- CMDR Series
- CMDR (
models/cmdr) - CMDR+ (
models/cmdr+) - CMDR Web (
models/cmdr-web) - CMDR+ Web (
models/cmdr+web)
- CMDR (
- Codestral (
models/codestral) - GPT-4 (
models/gpt-4,models/gpt-4o,models/gpt-4o-mini) - LLaVA 1.5 7B (
models/llava-1.5-7b-hf) - LZLV 70B (
models/lzlv-70b) - Mistral 7B (
models/mistral-7b) - PPLX 70B (
models/pplx-70b) - Qwen 72B (
models/qwen-72b) - Sonar Medium (
models/sonar-medium) - Striped Hyena
- Base (
models/stripedhyena) - Hessian (
models/stripedhyena-hessian)
- Base (
- Yi 34B (
models/yi-34b)
-h, --help Print help
-s, --settings <file> Load YLLM_* settings
-m, --model <model> Select model
-D, --set-default <model> Set default model
-S, --show-default Show default model
-C, --clear-default Clear default model
-a, --api-url <url> API URL
-k, --api-key <key> API key
-t, --temperature <t> Temperature (default: 0.1)
-p, --top-p <p> Top-p value (default: 0.9)
-l, --max-tokens <n> Max tokens (default: 4096)
-c, --stdin Read from stdin
-u, --url <url> Read from URL
-f, --file <file> Read from file
-d, --dump-prompt Show prompt and exit
-r, --raw-stream Show raw API stream
-z, --just-save-it Save to current directory
-Z, --save-it-to <prefix> Save with prefix
-P, --pdf-it Convert output to PDFInstall all required dependencies on Ubuntu/Debian with:
sudo apt install curl lynx jq file poppler-utils pandoc coreutilsDependencies and their purposes:
curl: API requestslynx: Web page extractionjq: JSON processingfile: File type detectionpdftotext(frompoppler-utils): PDF extractionpandoc: Format conversionstdbuf(fromcoreutils): Output buffering
MIT License - See LICENSE file