8000 Redistribute distribution events occurred after snapshot was taken · Issue #31871 · camunda/camunda · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

Redistribute distribution events occurred after snapshot was taken #31871

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
1 task
entangled90 opened this issue May 9, 2025 · 0 comments
Open
1 task

Redistribute distribution events occurred after snapshot was taken #31871

entangled90 opened this issue May 9, 2025 · 0 comments
Assignees
@panagiotisgts

Comments

@entangled90
Copy link
Contributor
entangled90 commented May 9, 2025 •
edited
Loading

Motivation

During the time when the snapshot is being taken, some distribution events may have been processed by the engine. They will not be part of the newly created snapshot and they are necessary for the correct processing in the new partition.

Implementation

Partition 1 - PartitionManagerImpl
When the snapshot is completed, we need to:

  • append a command to save the reservation_id of the snapshot in the state (hint: RedistributionState it's probably a good place)
    • there's no unique id that identifies the current scale up operation, but desiredPartitionCount should be enough
  • append a command to redistribute events from snapshot_idx to now.

When all events are redistributed [To be clarified if it needs to be done after the events are redistributed or after sending the command]:

  • append a follow-up event to include the new partition in the RoutingState used for regular distribution

Dependencies:
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@panagiotisgts panagiotisgts

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@entangled90 @panagiotisgts
0