8000 [Nested] Optimize structs in `LIST_VALUE` by maiadegraaf · Pull Request #17169 · duckdb/duckdb · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

[Nested] Optimize structs in LIST_VALUE #17169

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 21 commits into from
Jun 19, 2025
Merged

[Nested] Optimize structs in LIST_VALUE #17169

merged 21 commits into from
Jun 19, 2025

Conversation

maiadegraaf
Copy link
Contributor

This is a follow-up to

This update focuses specifically on optimizing performance when constructing LIST_VALUE with STRUCT elements.

Speed up examples:

Large Struct Table (Benchmark added in this PR):

CREATE TABLE large_struct_table AS
SELECT {'a': i - 5, 'b': i - 4} AS a, {'a': i - 3, 'b': i - 2} AS b, {'a': i - 1,'b': i} AS c FROM range(10000000) tbl(i);

SELECT LIST_VALUE(a, b, c) FROM large_struct_table;
v1.2.2 New
5.83 s 0.133 s

Nested Unpivot

CREATE TABLE structs AS
SELECT i AS id, {'id': i} AS "2020", {'id': i + 10} AS "2021", {'id': i + 100} AS "2022", {'id': i + 1000} AS "2023", {'id': i + 10000} AS "2024"
FROM range(10000000) t(i)

UNPIVOT structs ON "2020", "2021", "2022", "2023", "2024"
v1.2.2 New
3.24 s 0.39 s

taniabogatsch
taniabogatsch reviewed Apr 18, 2025
View reviewed changes
Copy link
Contributor
@taniabogatsch taniabogatsch left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @maiadegraaf, looks good! I left a few small comments.

@duckdb-draftbot duckdb-draftbot marked this pull request as draft April 22, 2025 13:11
@Tishj Tishj mentioned this pull request May 7, 2025
Merged
Mytherin added a commit that referenced this pull request May 8, 2025
@Mytherin
[Dev] Flatten any deeper children vectors, when the top level is a FL…
d4816b0 
…AT vector (#17387)

This PR fixes an issue found in
<#17169>

But since it has (likely) effect on much more tests and codepaths, I'm
making this PR separately.
@maiadegraaf maiadegraaf marked this pull request as ready for review May 8, 2025 17:04
@maiadegraaf maiadegraaf marked this pull request as draft May 9, 2025 14:14
@maiadegraaf maiadegraaf marked this pull request as ready for review May 9, 2025 14:14
Mytherin
Mytherin reviewed May 12, 2025
View reviewed changes
@duckdb-draftbot duckdb-draftbot marked this pull request as draft May 14, 2025 14:49
Tishj
Tishj reviewed May 14, 2025
View reviewed changes
@maiadegraaf maiadegraaf marked this pull request as ready for review May 23, 2025 14:45
@maiadegraaf
Copy link
Contributor Author

This PR has undergone further changes and now lists of structs are recursively built through calls to PopulateChild. I've also added even more tests

@duckdb-draftbot duckdb-draftbot marked this pull request as draft May 23, 2025 14:53
@maiadegraaf maiadegraaf marked this pull request as ready for review May 23, 2025 14:54
@duckdb-draftbot duckdb-draftbot marked this pull request as draft May 27, 2025 12:15
@maiadegraaf maiadegraaf marked this pull request as ready for review May 27, 2025 12:17
@duckdb-draftbot duckdb-draftbot marked this pull request as draft May 27, 2025 15:46
@maiadegraaf maiadegraaf force-pushed the list_value-struct-optimize branch from c8dcc4c to b83e375 Compare May 28, 2025 08:53
@maiadegraaf maiadegraaf force-pushed the list_value-struct-optimize branch 2 times, most recently from bc36c3b to a278694 Compare June 3, 2025 11:19
@maiadegraaf maiadegraaf force-pushed the list_value-struct-optimize branch from a278694 to b14a986 Compare June 3, 2025 13:47
@maiadegraaf maiadegraaf marked this pull request as ready for review June 6, 2025 10:24
@duckdb-draftbot duckdb-draftbot marked this pull request as draft June 11, 2025 15:57
@maiadegraaf maiadegraaf force-pushed the list_value-struct-optimize branch from e53a46f to b14a986 Compare June 17, 2025 13:32
@maiadegraaf maiadegraaf marked this pull request as ready for review June 17, 2025 13:37
@Mytherin Mytherin merged commit 360af14 into duckdb:main Jun 19, 2025
50 checks passed
@Mytherin
Copy link
Collaborator

Thanks!

@maiadegraaf maiadegraaf deleted the list_value-struct-optimize branch June 19, 2025 12:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@Mytherin Mytherin Mytherin left review comments

@Tishj Tishj Tishj left review comments

@taniabogatsch taniabogatsch taniabogatsch approved these changes

Assignees
No one assigned
Labels
Ready For Review
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@maiadegraaf @Mytherin @Tishj @taniabogatsch
0