vllm-project vllm · Discussions · GitHub

8000 vllm-project vllm · Discussions · GitHub

More Web Proxy on the site http://driver.im/

GitHub discussion is not used anymore, please use the forum for discussion.
Announcements youkaichao

Sort by: Latest activity

Categories

Discussions

You must be logged in to vote

Why is the GPU KV cache usage very low?
usage How to use vllm
tammypi announced Jun 18, 2024 in Q&A

3
You must be logged in to vote

Help serving vllm with kuberay

torsteinelv announced Jan 11, 2025 in Q&A

7
You must be logged in to vote

is paged attention an exact attention

CharlesJu1 announced Feb 26, 2025 in General · Closed

5
You must be logged in to vote

~50% GPU utilization of openai API server

imoneoi announced Jul 11, 2023 in Q&A

7
You must be logged in to vote

Plugins and addons

carelesswhisp announced Mar 20, 2025 in Q&A

2
You must be logged in to vote

Structured Generation with Reasoning Parser in offline mode.

psych0v0yager announced May 4, 2025 in General

1
You must be logged in to vote

Reasoning models and structured output (e.g. QwQ-32B and JSON)

matthew-at-qamcom announced Mar 27, 2025 in Q&A · Closed

4
You must be logged in to vote

How TOKENIZERS_PARALLELISM will affect vLLM?

Junjie-Chu announced Mar 20, 2024 in Q&A

1
You must be logged in to vote

0.7.4 开发版CPU使用率一直100%，即使没处理请求的时候也一样

AndrewTsao announced Mar 14, 2025 in General · Closed

1
You must be logged in to vote

is there a way to run vllm without torch.compiled model?

carlesoctav announced Dec 10, 2024 in Q&A

1
You must be logged in to vote

Running Llama4 quantized on 2xH100 80GB

ilyabcodin announced May 4, 2025 in Q&A · Closed

0
You must be logged in to vote

......lib/python3.12/site-packages/vllm/_C.abi3.so: undefined symbol: _ZN5torch3jit17parseSchemaOrNameERKSsb

RexLyu announced May 1, 2025 in Q&A

0
You must be logged in to vote

vLLM cannot connect to existing Ray cluster

as-bain announced May 1, 2025 in General

0
You must be logged in to vote

Determining Overall Speed for One Long Prompt

chigkim announced Apr 30, 2025 in Q&A

0
You must be logged in to vote

WARNING 02-09 10:24:45 cpu.py:99] ray is not supported on CPU, fallback to mp distributed executor backend.

jdmedeiros announced Feb 9, 2025 in Q&A

2
You must be logged in to vote

v0.8.2 "No available memory for the cache blocks." when launching models on the same GPU

CaoChensy announced Apr 1, 2025 in Q&A

3
You must be logged in to vote

Why gfx908 (mi100) was deleted from the arch list?

hnhyzz announced Apr 28, 2025 in General

0
You must be logged in to vote

Any known integration with n8n ?

wugutech announced Apr 28, 2025 in Q&A

0
You must be logged in to vote

I just published a performance test result of vllm vs sglang but can someone help me explain it?

qiulang announced Apr 26, 2025 in General · Closed

1
You must be logged in to vote

GitHub discussion is not used anymore, please use the forum for discussion.

youkaichao announced Mar 20, 2025 in Announcements

4
You must be logged in to vote

How do let vllm v1 to use a customized stat logger extended from StatLoggerBase?

liuzijing2014 announced Mar 5, 2025 in Q&A

1
You must be logged in to vote

Does vllm support CPU?

gokul427 announced Sep 9, 2023 in Q&A · Closed

7
You must be logged in to vote

Why temperature=0,top_p=1,seed=42 is still not enough 8DE8 to fix the llm's output!?

beyondguo announced Apr 25, 2025 in Q&A

0
You must be logged in to vote

LoRA Adapter enabling with vLLM is not working

SSPRATAP17 announced Apr 14, 2025 in Q&A

3
You must be logged in to vote

KV cache in VLLM serve

farouk09 announced Apr 23, 2025 in Q&A

0

0