8000 fix: set correct parent_hash for each kv block when publish kv events by ziqif-nv · Pull Request #671 · ai-dynamo/dynamo · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

fix: set correct parent_hash for each kv block when publish kv events #671

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 2 commits into from
Apr 15, 2025
Merged

fix: set correct parent_hash for each kv block when publish kv events #671

merged 2 commits into from
Apr 15, 2025

Conversation

ziqif-nv
Copy link
Contributor
@ziqif-nv ziqif-nv commented Apr 15, 2025 •
edited
Loading

Overview:

The correct parent_hash of each kv block should be from trtllm kv event.blocks.block_hash of the previous block in the sequence, except the first block, which is carried by trtllm kv event.data.parent_hash. However, currently, we set all blocks' parent_hash with event.data.parent_hash

Test:

With the change, sending the same requests would give high prefix hit rate, while before, the hit rate is quite low

2025-04-15T00:11:37.382Z  INFO processor._generate: Worker ID: 7587886108854196461 with estimated prefix hit rate: 0.9948186528497409
2025-04-15T00:11:39.746Z  INFO processor._generate: Worker ID: 7587886108854196461 with estimated prefix hit rate: 0.9948186528497409
2025-04-15T00:11:41.198Z  INFO processor._generate: Worker ID: 7587886108854196461 with estimated prefix hit rate: 0.9948186528497409

Related Issues: (use one of the action keywords Closes / Fixes / Resolves / Relates to)

@copy-pr-bot copy-pr-bot
Copy link
copy-pr-bot bot commented Apr 15, 2025

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@ziqif-nv ziqif-nv mentioned this pull request Apr 15, 2025
Closed
@ziqif-nv ziqif-nv enabled auto-merge (squash) April 15, 2025 00:26
@ziqif-nv ziqif-nv mentioned this pull request Apr 15, 2025
Open
4 tasks
@ziqif-nv ziqif-nv merged commit 15455df into main Apr 15, 2025
5 checks passed
@ziqif-nv ziqif-nv deleted the ziqif/fix-trtllm-parent-hash branch April 15, 2025 01:04
tanmayv25 pushed a commit that referenced this pull request Apr 15, 2025
@ziqif-nv @tanmayv25
fix: set correct parent_hash for each kv block when publish kv events (…
7ffc5c3 
…#671)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@tanmayv25 tanmayv25 tanmayv25 approved these changes

@hutm hutm Awaiting requested review from hutm hutm is a code owner

@biswapanda biswapanda Awaiting requested review from biswapanda biswapanda is a code owner

@ishandhanani ishandhanani Awaiting requested review from ishandhanani ishandhanani is a code owner

@julienmancuso julienmancuso Awaiting requested review from julienmancuso julienmancuso is a code owner

@hhzhang16 hhzhang16 Awaiting requested review from hhzhang16 hhzhang16 is a code owner

@nnshah1 nnshah1 Awaiting requested review from nnshah1 nnshah1 is a code owner

@piotrm-nvidia piotrm-nvidia Awaiting requested review from piotrm-nvidia piotrm-nvidia is a code owner

@ptarasiewiczNV ptarasiewiczNV Awaiting requested review from ptarasiewiczNV ptarasiewiczNV is a code owner

@ryanolson ryanolson Awaiting requested review from ryanolson ryanolson is a code owner

@grahamking grahamking Awaiting requested review from grahamking grahamking is a code owner

@paulhendricks paulhendricks Awaiting requested review from paulhendricks paulhendricks is a code owner

@tmonty12 tmonty12 Awaiting requested review from tmonty12 tmonty12 is a code owner

@GuanLuo GuanLuo Awaiting requested review from GuanLuo GuanLuo is a code owner

@rmccorm4 rmccorm4 Awaiting requested review from rmccorm4 rmccorm4 is a code owner

@tedzhouhk tedzhouhk Awaiting requested review from tedzhouhk tedzhouhk is a code owner

@alec-flowers alec-flowers Awaiting requested review from alec-flowers alec-flowers is a code owner

@kkranen kkranen Awaiting requested review from kkranen kkranen is a code owner

@sshchoi sshchoi Awaiting requested review from sshchoi sshchoi is a code owner

@PeaBrane PeaBrane Awaiting requested review from PeaBrane PeaBrane is a code owner

@mohammedabdulwahhab mohammedabdulwahhab Awaiting requested review from mohammedabdulwahhab mohammedabdulwahhab is a code owner

Assignees
No one assigned
Labels
size/XS
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@ziqif-nv @tanmayv25
0