8000 Refactor pgcopydb for standby server support by gokhangulbiz · Pull Request #655 · dimitri/pgcopydb · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

Refactor pgcopydb for standby server support #655

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 47 commits into from
May 30, 2024
Merged

Refactor pgcopydb for standby server support #655

merged 47 commits into from
May 30, 2024

Conversation

gokhangulbiz
Copy link
Contributor
@gokhangulbiz gokhangulbiz commented Jan 26, 2024 •
edited
Loading

This pull request refactors the pgcopydb to add support for running on a standby server. It includes changes to the copydb_export_snapshot() function to handle standby server mode by setting isolation level and readOnly mode properly and creates a new test case named pagila-standby in the GHA workflow.

Please note that, with this PR, pgcopydb skips partitioning on standby servers (where pg_is_in_recovery() is true) because it cannot execute ANALYZE on a read-only database.

@gokhangulbiz gokhangulbiz marked this pull request as draft January 26, 2024 14:22
dimitri
dimitri reviewed Jan 26, 2024
View reviewed changes
Copy link
Owner
@dimitri dimitri left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for this work @gokhangulbiz !

I see the test is failing because of the pgcopydb table size table creation. I think we could also just run a SELECT query to fetch the information and populate our SQLite catalogs with the result of it, in a new table s_table_size on the source catalogs. This would need to be done in yet another PR.

@arajkumar arajkumar mentioned this pull request Feb 1, 2024
Open
arajkumar
arajkumar reviewed Feb 6, 2024
View reviewed changes
@gokhangulbiz
Merge remote-tracking branch 'dimitri' into table-size-refactorings
7fb87c1
@gokhangulbiz
Add function to catalog.h for adding source table size
fc7c3c9
@gokhangulbiz
Refactor table size calculation in schema preparation
61be05a
@gokhangulbiz
Refactor partition calculation in schema preparation
4031f6d
@gokhangulbiz
Fix formatting in listSourceTableSizeSQL queries
79e8ad3
@gokhangulbiz
Fix regressions
8376b0d
@gokhangulbiz
Fix regressions
8962935
@gokhangulbiz
Merge remote-tracking branch 'dimitri' into table-size-refactorings
d5b5242
@gokhangulbiz
Merge remote-tracking branch 'dimitri' into pagila-standby
e526335
@gokhangulbiz
Merge branch 'table-size-refactorings' into pagila-standby
6268fd5
@gokhangulbiz
Add isReadOnly flag to TransactionSnapshot struct
668a08f
@gokhangulbiz
Initalize isReadOnly flag during snapshot operations
ad19019
@gokhangulbiz
Skip same table concurrency for readonly source servers
7ff86a3
@gokhangulbiz
Skip vacuuming for readonly source databases
47a5a5e
@gokhangulbiz
Add missing function comment for catalog_add_s_table_size
e03089b
@gokhangulbiz
Update relpages in s_table after analyze
8d9e80e
@gokhangulbiz
Refactor getPartKeyMinMaxValue function
817fa2d
@gokhangulbiz
Add missing function comment for getTableSizeArray(..)
d474dee
@gokhangulbiz
Add missing function comment for parseCurrentSourceTableSize(..)
2df11a6
@gokhangulbiz
Add comment for ctid based partitoning
be17dfc
@gokhangulbiz
Add comment for partition calculation
a2733d2
@gokhangulbiz
Fix build break
2e54a11
@gokhangulbiz
Fix schema_list_relpages return value
d99e585
@gokhangulbiz
Add missing function comment for catalog_add_s_table_size
bad06f5
@gokhangulbiz
Update relpages in s_table after analyze
52a0b01
@gokhangulbiz
Refactor getPartKeyMinMaxValue function
32b403d
@gokhangulbiz
Add missing function comment for getTableSizeArray(..)
0cbda55
@gokhangulbiz
Add missing function comment for parseCurrentSourceTableSize(..)
8ed4926
@gokhangulbiz
Add comment for ctid based partitoning
9cb18a4
@gokhangulbiz
Add comment for partition calculation
78fc1a6
@gokhangulbiz
Fix build break
c14c2b7
@gokhangulbiz
Fix schema_list_relpages return value
336ee9b
@gokhangulbiz
Merge branch 'table-size-refactorings' into pagila-standby
b398aca
@gokhangulbiz gokhangulbiz marked this pull request as ready for review February 29, 2024 05:58
@gokhangulbiz gokhangulbiz requested a review from dimitri February 29, 2024 05:58
dimitri
dimitri requested changes Mar 1, 2024
View reviewed changes
Copy link
Owner
@dimitri dimitri left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This now needs a rebase on-top of the current main branch.

dimitri
dimitri requested changes May 30, 2024
View reviewed changes
@dimitri dimitri merged commit 120601c into dimitri:main May 30, 2024
19 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@arajkumar arajkumar arajkumar left review comments

@dimitri dimitri dimitri approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@gokhangulbiz @dimitri @arajkumar
0