8000 Multi File Reader Rework (Part 19): Make `MultiFileReaderInterface` virtual, and move reading methods to the `BaseFileReader` by Mytherin · Pull Request #17475 · duckdb/duckdb · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

Multi File Reader Rework (Part 19): Make MultiFileReaderInterface virtual, and move reading methods to the BaseFileReader #17475

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 18 commits into from
May 14, 2025
Merged

Multi File Reader Rework (Part 19): Make MultiFileReaderInterface virtual, and move reading methods to the BaseFileReader #17475

merged 18 commits into from
May 14, 2025

Conversation

Mytherin
Copy link
Collaborator

This PR reworks the multi-file reader so that the MultiFileReaderInterface is a virtual class that must be inherited from instead of a class filled with static methods. This has a number of advantages:

  • The interface is well defined (virtual methods that must be implemented)
  • We can provide default implementations for methods that might not need to be overloaded by all implementations
  • This is extensible - we can add methods without needing to implement them to all implementations by providing a default

The interface is obtained through a single static method that must be defined on the interface object:

static unique_ptr<MultiFileReaderInterface> InitializeInterface(ClientContext &context, MultiFileReader &reader, MultiFileList &file_list);

Several methods are also moved from the MultiFileInterface to the BaseFileReader. This allows potentially different types of BaseFileReader objects to be used within the same multi-file reader.

Mytherin added 18 commits May 13, 2025 11:02
@duckdb-draftbot duckdb-draftbot marked this pull request as draft May 14, 2025 07:27
@Mytherin Mytherin marked this pull request as ready for review May 14, 2025 07:27
@Mytherin Mytherin merged commit 0671c70 into duckdb:main May 14, 2025
50 checks passed
krlmlr added a commit to duckdb/duckdb-r that referenced this pull request May 18, 2025
@krlmlr
vendor: Update vendored sources to duckdb/duckdb@0671c70
6f67344 
Multi File Reader Rework (Part 19): Make `MultiFileReaderInterface` virtual, and move reading methods to the `BaseFileReader` (duckdb/duckdb#17475)
Remove duplicated arrow fetch test (duckdb/duckdb#17476)
krlmlr added a commit to duckdb/duckdb-r that referenced this pull request May 19, 2025
@krlmlr
vendor: Update vendored sources to duckdb/duckdb@0671c70
5db9349 
Multi File Reader Rework (Part 19): Make `MultiFileReaderInterface` virtual, and move reading methods to the `BaseFileReader` (duckdb/duckdb#17475)
Remove duplicated arrow fetch test (duckdb/duckdb#17476)
krlmlr added a commit to duckdb/duckdb-r that referenced this pull request May 19, 2025
@krlmlr
vendor: Update vendored sources to duckdb/duckdb@0671c70
7dd3d10 
Multi File Reader Rework (Part 19): Make `MultiFileReaderInterface` virtual, and move reading methods to the `BaseFileReader` (duckdb/duckdb#17475)
Remove duplicated arrow fetch test (duckdb/duckdb#17476)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
1 participant
@Mytherin
0