8000 Improve execution scavenger by yux0 · Pull Request #3674 · temporalio/temporal · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

Improve execution scavenger #3674

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Nov 30, 2022
Merged

Improve execution scavenger #3674

merged 3 commits into from
Nov 30, 2022

Conversation

yux0
Copy link
Contributor
@yux0 yux0 commented Nov 29, 2022

What changed?

  1. Skip history validation if the mutable state validation failed.
  2. Improve the execution scavenger task to continue iterate all records if error encounters.

Why?

  1. fail fast if mutable state is corrupted. It indicates bugs or DB issues.
  2. Improve the efficiency to scan all records.

How did you test it?

Potential risks

Is hotfix candidate?
No

@yux0 yux0 requested a review from a team as a code owner November 29, 2022 19:56
@@ -167,6 +181,11 @@ func (t *task) validate(
results = append(results, validationResults...)
}

// Fail fast if the mutable is corrupted, no need to validate history.
if len(results) > 0 {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

shall we change the mutableStateValidator.Validate() logic to check retention first and if it exceeds retention it should not do any further checking. Also we should enable delete when it detect closed executions exceeds retention.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes. Currently, it will delete if expired data detected.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Update to return if the retention check fails.

@yux0 yux0 enabled auto-merge (squash) November 29, 2022 23:59
@yux0 yux0 merged commit 0101924 into temporalio:master Nov 30, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants
0