8000 prototype log version 8.0 upgrades/downgrades by dlambrig · Pull Request #12141 · apple/foundationdb · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

prototype log version 8.0 upgrades/downgrades #12141

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 1 commit into
base: version-vector-disk-queue
Choose a base branch
from
Open

prototype log version 8.0 upgrades/downgrades #12141

wants to merge 1 commit into from

Conversation

dlambrig
Copy link
Contributor
@dlambrig dlambrig commented May 9, 2025 •
edited
Loading

This PR introduces a new tLog version 8.0 for the changes to the disk queue for version vector encoded in PR11777. Per the upgrade procedure described in the design document tlog-forward-compatibility, the fdb version in this proof-of-concept PR is increased to 7.5.0. To modify the tlog's disk format, the document describes the first two steps below. To enable version vector, a third step would set the vv feature to enabled.

  1. upgrade the fdb version to 7.5.0 which has both the old tlog sever code (OldTLogServer_7_0.actor.cpp) and the new one. This is in fdbserver version 7.5.0. Because the log version is still set to the old value 7.0, the old code in OldTLogServer_7_0.actor.cpp is run.
  2. upgrade the log version to the new one using fdbcli> configure log_version:=8. This will incur a restart which will run the code in the new TLogServer.actor.cpp with the changes in PR11777. The new disk queue entries prevVersion and tLogLocIds will then be written on commits. Because version vector is still disabled the value of tLogLocIds is empty.
  3. Version vector can now be enabled. When the new tlog recovery is run the disk queue entries generated in step 2 will be read with dummy values. (discussion pending)

As the document tlog-forward-compatibility.md.html describes, a recovering shared tlog could have log data instances with underlying disk queue files in both previous and new formats (the log version is encoded in the file name). This could happen in step 2. The corresponding OldTLogServer_7_0 or new TLogServer will process their disk queues according to their log versions.

A downgrade would run these steps in reverse order.

This PR is a proof of concept.

Code-Reviewer Section

The general pull request guidelines can be found here.

Please check each of the following things and check all boxes before accepting a PR.

  • The PR has a description, explaining both the problem and the solution.
  • The description mentions which forms of testing were done and the testing seems reasonable.
  • Every function/class/actor that was touched is reasonably well documented.

For Release-Branches

If this PR is made against a release-branch, please also check the following:

  • This change/bugfix is a cherry-pick from the next younger branch (younger release-branch or main i 8000 f this is the youngest branch)
  • There is a good reason why this PR needs to go into a release branch and this reason is documented (either in the description above or in a linked GitHub issue)

@dlambrig dlambrig requested review from jzhou77 and sbodagala May 9, 2025 15:11
@foundationdb-ci
Copy link
Contributor

Result of foundationdb-pr-clang-arm on Linux CentOS 7

  • Commit ID: ca3db90
  • Duration 0:04:31
  • Result: ❌ FAILED
  • Error: Error while executing command: if [[ $(git diff --shortstat 2> /dev/null | tail -n1) == "" ]]; then echo "CODE FORMAT CLEAN"; else echo "CODE FORMAT NOT CLEAN"; echo; echo "THE FOLLOWING FILES NEED TO BE FORMATTED"; echo; git ls-files -m; echo; if [[ $FDB_VERSION =~ 7\.\3. ]]; then echo skip; else exit 1; fi; fi. Reason: exit status 1
  • Build Log terminal output (available for 30 days)
  • Build Workspace zip file of the working directory (available for 30 days)

@foundationdb-ci
Copy link
Contributor

Result of foundationdb-pr-clang on Linux RHEL 9

  • Commit ID: ca3db90
  • Duration 0:04:33
  • Result: ❌ FAILED
  • Error: Error while executing command: if [[ $(git diff --shortstat 2> /dev/null | tail -n1) == "" ]]; then echo "CODE FORMAT CLEAN"; else echo "CODE FORMAT NOT CLEAN"; echo; echo "THE FOLLOWING FILES NEED TO BE FORMATTED"; echo; git ls-files -m; echo; if [[ $FDB_VERSION =~ 7\.\3. ]]; then echo skip; else exit 1; fi; fi. Reason: exit status 1
  • Build Log terminal output (available for 30 days)
  • Build Workspace zip file of the working directory (available for 30 days)

@foundationdb-ci
Copy link
Contributor

Result of foundationdb-pr-cluster-tests on Linux RHEL 9

  • Commit ID: ca3db90
  • Duration 0:04:41
  • Result: ❌ FAILED
  • Error: Error while executing command: if [[ $(git diff --shortstat 2> /dev/null | tail -n1) == "" ]]; then echo "CODE FORMAT CLEAN"; else echo "CODE FORMAT NOT CLEAN"; echo; echo "THE FOLLOWING FILES NEED TO BE FORMATTED"; echo; git ls-files -m; echo; if [[ $FDB_VERSION =~ 7\.\3. ]]; then echo skip; else exit 1; fi; fi. Reason: exit status 1
  • Build Log terminal output (available for 30 days)
  • Build Workspace zip file of the working directory (available for 30 days)
  • Cluster Test Logs zip file of the test logs (available for 30 days)
8000

@foundationdb-ci
Copy link
Contributor

Result of foundationdb-pr on Linux RHEL 9

  • Commit ID: ca3db90
  • Duration 0:04:45
  • Result: ❌ FAILED
  • Error: Error while executing command: if [[ $(git diff --shortstat 2> /dev/null | tail -n1) == "" ]]; then echo "CODE FORMAT CLEAN"; else echo "CODE FORMAT NOT CLEAN"; echo; echo "THE FOLLOWING FILES NEED TO BE FORMATTED"; echo; git ls-files -m; echo; if [[ $FDB_VERSION =~ 7\.\3. ]]; then echo skip; else exit 1; fi; fi. Reason: exit status 1
  • Build Log terminal output (available for 30 days)
  • Build Workspace zip file of the working directory (available for 30 days)

@dlambrig dlambrig changed the title prototype log version 8.0 prototype log version 8.0 upgrades/downgrades May 9, 2025
@foundationdb-ci
Copy link
Contributor

Result of foundationdb-pr-macos-m1 on macOS Ventura 13.x

  • Commit ID: ca3db90
  • Duration 0:19:58
  • Result: ❌ FAILED
  • Error: Error while executing command: ssh -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -i ${HOME}/.ssh_key ec2-user@${MAC_EC2_HOST} /opt/homebrew/bin/bash --login -c ./build_pr_macos.sh. Reason: exit status 1
  • Build Log terminal output (available for 30 days)
  • Build Workspace zip file of the working directory (available for 30 days)

@foundationdb-ci
Copy link
Contributor

Result of foundationdb-pr-macos on macOS Ventura 13.x

  • Commit ID: ca3db90
  • Duration 0:27:05
  • Result: ❌ FAILED
  • Error: Error while executing command: ssh -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -i ${HOME}/.ssh_key ec2-user@${MAC_EC2_HOST} /usr/local/bin/bash --login -c ./build_pr_macos.sh. Reason: exit status 1
  • Build Log terminal output (available for 30 days)
  • Build Workspace zip file of the working directory (available for 30 days)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@jzhou77 jzhou77 Awaiting requested review from jzhou77

@sbodagala sbodagala Awaiting requested review from sbodagala

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@dlambrig @foundationdb-ci
0