8000 fix: should route based on waiting requests, not active by PeaBrane · Pull Request #989 · ai-dynamo/dynamo · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

fix: should route based on waiting requests, not active #989

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 2 commits into from
May 8, 2025
Merged

fix: should route based on waiting requests, not active #989

merged 2 commits into from
May 8, 2025

Conversation

PeaBrane
Copy link
Contributor
@PeaBrane PeaBrane commented May 7, 2025

Overview:

To be in sync with the Python routing cost function:

  1. should update and route based on num_requests_waiting not num_active_requests
  2. best_logit = 0 should not error, as it is common for the first couple requests, and also possible if other requests are negative in logits

@copy-pr-bot copy-pr-bot
Copy link
copy-pr-bot bot commented May 7, 2025

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@PeaBrane PeaBrane merged commit 8bdf18e into main May 8, 2025
9 checks passed
@PeaBrane PeaBrane deleted the rupei/rusty-router-fixes branch May 8, 2025 16:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@GuanLuo GuanLuo GuanLuo approved these changes

@ryanolson ryanolson Awaiting requested review from ryanolson ryanolson is a code owner

@grahamking grahamking Awaiting requested review from grahamking grahamking is a code owner

@paulhendricks paulhendricks Awaiting requested review from paulhendricks paulhendricks is a code owner

@biswapanda biswapanda Awaiting requested review from biswapanda biswapanda is a code owner

@tmonty12 tmonty12 Awaiting requested review from tmonty12 tmonty12 is a code owner

@rmccorm4 rmccorm4 Awaiting requested review from rmccorm4 rmccorm4 is a code owner

@alec-flowers alec-flowers Awaiting requested review from alec-flowers alec-flowers is a code owner

@kkranen kkranen Awaiting requested review from kkranen kkranen is a code owner

@oandreeva-nv oandreeva-nv Awaiting requested review from oandreeva-nv oandreeva-nv is a code owner

@jthomson04 jthomson04 Awaiting requested review from jthomson04 jthomson04 is a code owner

Assignees
No one assigned
Labels
size/S
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@PeaBrane @GuanLuo
0