8000 feat(sglang): aggregated support by ishandhanani · Pull Request #937 · ai-dynamo/dynamo · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

feat(sglang): aggregated support #937

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 32 commits into from
May 9, 2025
Merged

feat(sglang): aggregated support #937

merged 32 commits into from
May 9, 2025

Conversation

ishandhanani
Copy link
Contributor
@ishandhanani ishandhanani commented May 4, 2025 •
edited
Loading

Add support for aggregated SGLang

Phases

  • agg chat/completions

closes DYN-238

Ubuntu and others added 2 commits May 4, 2025 22:43
@copy-pr-bot copy-pr-bot
Copy link
copy-pr-bot bot commented May 4, 2025

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@ishandhanani
Copy link
Contributor Author
ishandhanani commented May 5, 2025 •
edited
Loading

streaming and non streaming works for tp1 model - have not tinkered with other models or settings yet

ishandhanani
ishandhanani commented May 5, 2025
View reviewed changes
ishandhanani added 2 commits May 5, 2025 16:19
@ishandhanani ishandhanani changed the title [WIP]: sglang support feat: aggregated sglang support May 5, 2025
@ishandhanani ishandhanani mentioned this pull request May 5, 2025
Closed
@ishandhanani ishandhanani changed the title feat: aggregated sglang support feat(sglang): aggregated support May 5, 2025
@ishandhanani ishandhanani marked this pull request as ready for review May 5, 2025 20:22
@ishandhanani ishandhanani requested a review from jthomson04 as a code owner May 6, 2025 20:28
@github-actions github-actions bot added the feat label May 8, 2025
tedzhouhk
tedzhouhk reviewed May 9, 2025
View reviewed changes
rmccorm4
rmccorm4 approved these changes May 9, 2025
View reviewed changes
rmccorm4
rmccorm4 reviewed May 9, 2025
View reviewed changes
@ishandhanani ishandhanani enabled auto-merge (squash) May 9, 2025 05:44
@ishandhanani ishandhanani merged commit 5d5235b into main May 9, 2025
8 checks passed
@ishandhanani ishandhanani deleted the ishan/sglang branch May 9, 2025 06:01
biswapanda pushed a commit that referenced this pull request May 9, 2025
@ishandhanani @biswapanda
feat(sglang): aggregated support (#937)
571fee1 
Co-authored-by: ishandhanani <ishandhananai@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@rmccorm4 rmccorm4 rmccorm4 approved these changes

@GuanLuo GuanLuo GuanLuo left review comments

@ptarasiewiczNV ptarasiewiczNV ptarasiewiczNV left review comments

@tedzhouhk tedzhouhk tedzhouhk approved these changes

@PeaBrane PeaBrane PeaBrane approved these changes

@hutm hutm Awaiting requested review from hutm hutm is a code owner

@biswapanda biswapanda Awaiting requested review from biswapanda biswapanda is a code owner

@julienmancuso julienmancuso Awaiting requested review from julienmancuso julienmancuso is a code owner

@hhzhang16 hhzhang16 Awaiting requested review from hhzhang16 hhzhang16 is a code owner

@nnshah1 nnshah1 Awaiting requested review from nnshah1 nnshah1 is a code owner

@tanmayv25 tanmayv25 Awaiting requested review from tanmayv25 tanmayv25 is a code owner

@piotrm-nvidia piotrm-nvidia Awaiting requested review from piotrm-nvidia piotrm-nvidia is a code owner

@ryanolson ryanolson Awaiting requested review from ryanolson ryanolson is a code owner

@grahamking grahamking Awaiting requested review from grahamking grahamking is a code owner

@paulhendricks paulhendricks Awaiting requested review from paulhendricks paulhendricks is a code owner

@tmonty12 tmonty12 Awaiting requested review from tmonty12 tmonty12 is a code owner

@alec-flowers alec-flowers Awaiting requested review from alec-flowers alec-flowers is a code owner

@kkranen kkranen Awaiting requested review from kkranen kkranen is a code owner

@sshchoi sshchoi Awaiting requested review from sshchoi sshchoi is a code owner

@mohammedabdulwahhab mohammedabdulwahhab Awaiting requested review from mohammedabdulwahhab mohammedabdulwahhab is a code owner

@jthomson04 jthomson04 Awaiting requested review from jthomson04 jthomson04 is a code owner

Assignees
No one assigned
Labels
feat size/L
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
7 participants
@ishandhanani @grahamking @tedzhouhk @rmccorm4 @GuanLuo @PeaBrane @ptarasiewiczNV
0