8000 docs: Guides for single node benchmarking by kthui · Pull Request #509 · ai-dynamo/dynamo · GitHub

More Web Proxy on the site http://driver.im/

docs: Guides for single node benchmarking #509

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Merged

kthui merged 10 commits into main from jacky-benchmarking-guide

Apr 9, 2025

+298 −0

Contributor

kthui commented

Overview:

Guides for single/two nodes benchmarking

Details:

N/A

Where should the reviewer start?

N/A

Related Issues: (use one of the action keywords Closes / Fixes / Resolves / Relates to)

N/A


          Guides for single/two nodes benchmarking

688dbc4

copy-pr-bot bot commented

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

kthui self-assigned this

kthui added the documentation label


          Enhance vLLM baseline instructions

6b9713d

piotrm-nvidia reviewed

View reviewed changes

examples/llm/benchmarking.md Outdated Show resolved Hide resolved


          Update benchmarking config

8fa7ee5

kthui force-pushed the jacky-benchmarking-guide branch from 85155c6 to 8fa7ee5 Compare

April 6, 2025 01:01

kthui marked this pull request as ready for review

April 6, 2025 01:07

kthui requested review from hutm, biswapanda, ishandhanani, julienmancuso, hhzhang16, nnshah1, tanmayv25, ptarasiewiczNV, ryanolson, grahamking, paulhendricks, tmonty12, GuanLuo, rmccorm4, tedzhouhk, alec-flowers, kkranen, sshchoi, PeaBrane, mohammedabdulwahhab and a team as code owners

April 6, 2025 01:07


          Include single node benchmarking currently

afac84c

kthui changed the title ~~docs: Guides for single/two nodes benchmarking~~ docs: Guides for single nodes benchmarking


          Merge branch 'main' into jacky-benchmarking-guide

935011f

kthui changed the title ~~docs: Guides for single nodes benchmarking~~ docs: Guides for single node benchmarking

nnshah1 reviewed

View reviewed changes

examples/llm/benchmarks/README.md Outdated Show resolved Hide resolved

nnshah1 reviewed

View reviewed changes

examples/llm/benchmarks/README.md Outdated Show resolved Hide resolved

nnshah1 reviewed

View reviewed changes

examples/llm/benchmarks/README.md Show resolved Hide resolved

nnshah1 reviewed

View reviewed changes

examples/llm/benchmarks/README.md Outdated Show resolved Hide resolved

nnshah1 reviewed

View reviewed changes

examples/llm/benchmarks/README.md Outdated Show resolved Hide resolved

nnshah1 reviewed

View reviewed changes

examples/llm/benchmarks/README.md Show resolved Hide resolved

nnshah1 reviewed

View reviewed changes

examples/llm/benchmarks/disagg.yaml Show resolved Hide resolved

nnshah1 reviewed

View reviewed changes

examples/llm/benchmarks/disagg.yaml Show resolved Hide resolved

nnshah1 reviewed

View reviewed changes

examples/llm/benchmarks/disagg.yaml Show resolved Hide resolved

nnshah1 reviewed

View reviewed changes

examples/llm/benchmarks/perf.sh Show resolved Hide resolved

nnshah1 approved these changes

View reviewed changes

8000

Contributor

nnshah1 left a comment

Left some items for future - let's remove the "coming soon" unless needed -

kthui and others added 2 commits

April 8, 2025 17:46


          Update examples/llm/benchmarks/README.md

44c0cd9

Co-authored-by: Neelay Shah <neelays@nvidia.com>
Signed-off-by: Jacky <18255193+kthui@users.noreply.github.com>


          Move key settings into its config file, and move coming soon to futur…

05ffd21

…e roadmap

kthui mentioned this pull request

docs: Guide for multi-node benchmarking #561

Merged

Contributor Author

kthui commented

Let us move all non-single-node setup discussions to this PR #561, because we want to merge in the single node guide soon if ready.

kthui merged commit 8516dca into main

6 checks passed

kthui deleted the jacky-benchmarking-guide branch

April 9, 2025 18:37

kthui added a commit that referenced this pull request


          docs: Guides for single node benchmarking (#509)

f7a5a3e

Signed-off-by: Jacky <18255193+kthui@users.noreply.github.com>
Co-authored-by: Piotr Marcinkiewicz <piotrm@nvidia.com>
Co-authored-by: Neelay Shah <neelays@nvidia.com>

kthui mentioned this pull request

docs: Guides for single node benchmarking (#509) #616

Merged

nv-anants pushed a commit that referenced this pull request


          docs: Guides for single node benchmarking (#509) (#616)

28f9697

Signed-off-by: Jacky <18255193+kthui@users.noreply.github.com>
Co-authored-by: Piotr Marcinkiewicz <piotrm@nvidia.com>
Co-authored-by: Neelay Shah <neelays@nvidia.com>

saturley-hall pushed a commit that referenced this pull request


          docs: Guides for single node benchmarking (#509) (#616)

Signed-off-by: Jacky <18255193+kthui@users.noreply.github.com>
Co-authored-by: Piotr Marcinkiewicz <piotrm@nvidia.com>
Co-authored-by: Neelay Shah <neelays@nvidia.com>

saturley-hall pushed a commit that referenced this pull request


          docs: Guides for single node benchmarking (#509)

288c425

Signed-off-by: Jacky <18255193+kthui@users.noreply.github.com>
Co-authored-by: Piotr Marcinkiewicz <piotrm@nvidia.com>
Co-authored-by: Neelay Shah <neelays@nvidia.com>

kylehh pushed a commit to kylehh/dynamo that referenced this pull request


          docs: Guides for single node benchmarking (ai-dynamo#509)

594f8c0

Signed-off-by: Jacky <18255193+kthui@users.noreply.github.com>
Co-authored-by: Piotr Marcinkiewicz <piotrm@nvidia.com>
Co-authored-by: Neelay Shah <neelays@nvidia.com>

This was referenced Apr 15, 2025

docs: Benchmarking guide updates #678

Merged

docs: Benchmarking guide interpreting results #701

Open

oandreeva-nv pushed a commit that referenced this pull request


          docs: Guides for single node benchmarking (#509)

791899b

Signed-off-by: Jacky <18255193+kthui@users.noreply.github.com>
Co-authored-by: Piotr Marcinkiewicz <piotrm@nvidia.com>
Co-authored-by: Neelay Shah <neelays@nvidia.com>

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

tedzhouhk tedzhouhk left review comments

nnshah1 nnshah1 approved these changes

piotrm-nvidia piotrm-nvidia left review comments

hutm Awaiting requested review from hutm hutm is a code owner

biswapanda Awaiting requested review from biswapanda biswapanda is a code owner

ishandhanani Awaiting requested review from ishandhanani ishandhanani is a code owner

julienmancuso Awaiting requested review from julienmancuso julienmancuso is a code owner

hhzhang16 Awaiting requested review from hhzhang16 hhzhang16 is a code owner

tanmayv25 Awaiting requested review from tanmayv25 tanmayv25 is a code owner

ptarasiewiczNV Awaiting requested review from ptarasiewiczNV ptarasiewiczNV is a code owner

ryanolson Awaiting requested review from ryanolson ryanolson is a code owner

grahamking Awaiting requested review from grahamking grahamking is a code owner

paulhendricks Awaiting requested review from paulhendricks paulhendricks is a code owner

tmonty12 Awaiting requested review from tmonty12 tmonty12 is a code owner

GuanLuo Awaiting requested review from GuanLuo GuanLuo is a code owner

rmccorm4 Awaiting requested review from rmccorm4 rmccorm4 is a code owner

alec-flowers Awaiting requested review from alec-flowers alec-flowers is a code owner

kkranen Awaiting requested review from kkranen kkranen is a code owner

sshchoi Awaiting requested review from sshchoi sshchoi is a code owner

PeaBrane Awaiting requested review from PeaBrane PeaBrane is a code owner

mohammedabdulwahhab Awaiting requested review from mohammedabdulwahhab mohammedabdulwahhab is a code owner

Labels

0