8000 Extended posgres Drop() with type dropping by kvij · Pull Request #627 · golang-migrate/migrate · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

Extended posgres Drop() with type dropping #627

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 5 commits into
base: master
Choose a base branch
from

Conversation

kvij
Copy link
@kvij kvij commented Sep 26, 2021

Closes #626

Fixes that Migrate.Drop() is not clearing the database properly in both the postgres and pgx database driver.

@kvij kvij changed the title Extended posgres Drop() with type droping Extended posgres Drop() with type dropping Sep 26, 2021
@coveralls
Copy link
coveralls commented Sep 26, 2021

Pull Request Test Coverage Report for Build 1459982313

  • 76 of 84 (90.48%) changed or added relevant lines in 2 files are covered.
  • No unchanged relevant lines lost coverage.
  • Overall coverage increased (+0.2%) to 57.978%

Changes Missing Coverage Covered Lines Changed/Added Lines %
database/pgx/pgx.go 38 42 90.48%
database/postgres/postgres.go 38 42 90.48%
Totals Coverage Status
Change from base Build 1424470324: 0.2%
Covered Lines: 3779
Relevant Lines: 6518

💛 - Coveralls

@kvij kvij force-pushed the 626-postrgres-drop-types branch from 8a96bb7 to 9159f93 Compare October 31, 2021 09:51
Copy link
Member
@dhui dhui left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the PR and sorry for the delayed review!
I like how this approach is cleaner than this PR's

switch element {
case tableSQL:
query = `SELECT table_name FROM information_schema.tables WHERE table_schema=(SELECT current_schema()) AND table_type='BASE TABLE'`
case typeSQL:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add tests for this. e.g. that this will only drop unused types in the current schema

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This function does not check for unused. Drop() however does what you expect it would. First it drops all the tables. Than it drops all the custom types. Because the tables are dropped first all the custom types are unused.
The expected result is now tested.

SELECT t.typname as type
FROM pg_type t
LEFT JOIN pg_catalog.pg_namespace n ON n.oid = t.typnamespace
WHERE (t.typrelid = 0 OR (SELECT c.relkind = 'c' FROM pg_catalog.pg_class c WHERE c.oid = t.typrelid))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you add comments for this query? It's not immediately clear to me what the filtering conditions are doing due to my lack of familiarity with the postgres system catalogs.

Docs:

Also, shouldn't

WHERE (t.typrelid = 0 OR (SELECT c.relkind = 'c' FROM pg_catalog.pg_class c WHERE c.oid = t.typrelid))

be

WHERE (t.typrelid = 0 OR (SELECT 1 FROM pg_catalog.pg_class c WHERE c.oid = t.typrelid AND c.relkind = 'c'))

Copy link
Author
@kvij kvij Nov 3, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the documentation I had some time to look into it a bit. I'm very new to postgresql so a few weeks before this PR I did not even know arrays where possible. I took this query from stackoverflow and adjusted it for the current schema and tested it. The source of the query I modified is actually psql \dT.

Also, shouldn't
WHERE (t.typrelid = 0 OR (SELECT c.relkind = 'c' FROM pg_catalog.pg_class c WHERE c.oid = t.typrelid))
be
WHERE (t.typrelid = 0 OR (SELECT 1 FROM pg_catalog.pg_class c WHERE c.oid = t.typrelid AND c.relkind = 'c'))

No the query is correct. Your suggestion will result in a syntax error (subquery needs to return a boolean). I don´t fully understand why this is needed because I'm unfamiliar with the administration of composite types. What this WHERE does is include all composite and non-composite types. Types can apparently also have a typerel of a table or index etc. In the source of psql is is described as

do not include complex types (typrelid!=0) unless they are standalone composite types.

I will do my best to formulate a comprehensive comment.

@dhui
Copy link
Member
dhui commented Oct 31, 2021

This will fix #193

@kvij
Copy link
Author
kvij commented Nov 2, 2021

Ah missed that issue. Created a duplicate.

Yes sometimes we don't care or need to care about the Down. Down is specifically only for things created by Up.
Down can be very slow a large database/complex reverts.
Using the Drop without this fix leaves the database in a state that cannot be fixed with migrate.

@kvij kvij force-pushed the 626-postrgres-drop-types branch 3 times, most recently from debb120 to bf9df6c Compare November 14, 2021 22:01
@kvij kvij force-pushed the 626-postrgres-drop-types branch from bf9df6c to 19eb878 Compare November 14, 2021 23:14
@kvij kvij requested a review from dhui November 18, 2021 18:08
@kvij
Copy link
Author
kvij commented Nov 28, 2021

@dhui Can you please have another look? I'm staring to get out of context again.

@kvij
Copy link
Author
kvij commented Jan 3, 2022

Hi @dhui could you please continue the review?

@Fontinalis Fontinalis added this to the v4.16.0 milestone Apr 27, 2022
@Fontinalis Fontinalis added the database Updates database drivers label Apr 27, 2022
@darylhjd
Copy link

just got bit by this. would be great if this could be reviewed and merged soon!

@stevenh
Copy link
stevenh commented Mar 9, 2024

Same here

@gbgil
Copy link
gbgil commented May 28, 2024

Also stumbled upon this issue caused by #193 and lost some minutes trying to figure out what was going on, this getting reviewed and merged would be very appreciated!

@fedemzcor
Copy link

merge? when?

@kvij
Copy link
Author
kvij commented Dec 6, 2024

merge? when?

Most likely never. I've tried it. Without commitment from maintainers I'm not rebasing.
Not looking for pointless work.

@mxfactorial
Copy link

substituting with psql drops types (dev instances only pls)

thank you for the effort @kvij

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
database Updates database drivers
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Postgres Drop() implementation incomplete
9 participants
0