8000 add workaround for patching httpfs ext by samansmink · Pull Request #16722 · duckdb/duckdb · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

add workaround for patching httpfs ext #16722

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 1 commit into from
Mar 19, 2025
Merged

add workaround for patching httpfs ext #16722

merged 1 commit into from
Mar 19, 2025

Conversation

samansmink
Copy link
Contributor

Somehow the FETCHCONTENT_POPULATE step fails in CI with no good way to debug this. I did not manage to repro this locally, and the error messages produced are completely useless:

See error 
  Successfully applied patch to D:/a/duckdb/duckdb/.github/patches/extensions/httpfs/:
  RAN git diff on D:\a\duckdb\duckdb\build\extension_configuration\_deps\httpfs_extension_fc-src
  No configure step for 'httpfs_extension_fc-populate'
  No build step for 'httpfs_extension_fc-populate'
  No install step for 'httpfs_extension_fc-populate'
  No test step for 'httpfs_extension_fc-populate'
  Completed 'httpfs_extension_fc-populate'
C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\MSBuild\Microsoft\VC\v160\Microsoft.CppCommon.targets(241,5): error MSB8066: Custom build for 'D:\a\duckdb\duckdb\build\extension_configuration\_deps\httpfs_extension_fc-subbuild\CMakeFiles\f3b9797c66ae368a3d3b142f0b600a26\httpfs_extension_fc-populate-download.rule;D:\a\duckdb\duckdb\build\extension_configuration\_deps\httpfs_extension_fc-subbuild\CMakeFiles\f3b9797c66ae368a3d3b142f0b600a26\httpfs_extension_fc-populate-update.rule;D:\a\duckdb\duckdb\build\extension_configuration\_deps\httpfs_extension_fc-subbuild\CMakeFiles\f3b9797c66ae368a3d3b142f0b600a26\httpfs_extension_fc-populate-patch.rule;D:\a\duckdb\duckdb\build\extension_configuration\_deps\httpfs_extension_fc-subbuild\CMakeFiles\f3b9797c66ae368a3d3b142f0b600a26\httpfs_extension_fc-populate-configure.rule;D:\a\duckdb\duckdb\build\extension_configuration\_deps\httpfs_extension_fc-subbuild\CMakeFiles\f3b9797c66ae368a3d3b142f0b600a26\httpfs_extension_fc-populate-build.rule;D:\a\duckdb\duckdb\build\extension_configuration\_deps\httpfs_extension_fc-subbuild\CMakeFiles\f3b9797c66ae368a3d3b142f0b600a26\httpfs_extension_fc-populate-install.rule;D:\a\duckdb\duckdb\build\extension_configuration\_deps\httpfs_extension_fc-subbuild\CMakeFiles\f3b9797c66ae368a3d3b142f0b600a26\httpfs_extension_fc-populate-test.rule;D:\a\duckdb\duckdb\build\extension_configuration\_deps\httpfs_extension_fc-subbuild\CMakeFiles\22bea71fcf663c1a2120766248011d84\httpfs_extension_fc-populate-complete.rule;D:\a\duckdb\duckdb\build\extension_configuration\_deps\httpfs_extension_fc-subbuild\CMakeFiles\c0e352f43e30213971dd8faa4d28848d\httpfs_extension_fc-populate.rule;D:\a\duckdb\duckdb\build\extension_configuration\_deps\httpfs_extension_fc-subbuild\CMakeLists.txt' exited with code -1. [D:\a\duckdb\duckdb\build\extension_configuration\_deps\httpfs_extension_fc-subbuild\httpfs_extension_fc-populate.vcxproj]

CMake Error at C:/Program Files/CMake/share/cmake-3.31/Modules/FetchContent.cmake:1918 (message):
Error:   Build step for httpfs_extension_fc failed: 1
Call Stack (most recent call first):
  C:/Program Files/CMake/share/cmake-3.31/Modules/FetchContent.cmake:1609 (__FetchContent_populateSubbuild)
  C:/Program Files/CMake/share/cmake-3.31/Modules/FetchContent.cmake:2145:EVAL:2 (__FetchContent_doPopulation)
  C:/Program Files/CMake/share/cmake-3.31/Modules/FetchContent.cmake:2145 (cmake_language)
  C:/Program Files/CMake/share/cmake-3.31/Modules/FetchContent.cmake:1978:EVAL:1 (__FetchContent_Populate)
  C:/Program Files/CMake/share/cmake-3.31/Modules/FetchContent.cmake:1978 (cmake_language)
  CMakeLists.txt:1099 (FETCHCONTENT_POPULATE)
  CMakeLists.txt:1208 (register_external_extension)
  .github/config/out_of_tree_extensions.cmake:19 (duckdb_extension_load)
  CMakeLists.txt:1298 (include)

This basically means that patching httpfs should be done rarely and patches should be applied upsteam asap since they will prevent nightly builds from using httpfs on windows

This was referenced Mar 19, 2025
Closed
Merged
carlopi
carlopi approved these changes Mar 19, 2025
View reviewed changes
Copy link
Contributor
@carlopi carlopi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agree, if this happens more frequently / in other cases, there might be some breadcrumbs, but for now all cool, this seems the reasonable way forward.

@Mytherin Mytherin merged commit ea7d259 into duckdb:main Mar 19, 2025
50 checks passed
@Mytherin
Copy link
Collaborator
< 8000 /tr>

Thanks!

@carlopi carlopi mentioned this pull request Apr 11, 2025
Merged
Mytherin added a commit that referenced this pull request Apr 11, 2025
@Mytherin
Fix httpfs patches: avoid git log since might contain unsanitised `…
aadcb3d 
…error` word (#17075)

Avoid `git log` while applying patches, that will print a bunch of
content to stdout.
The stdout then might (or not) contain the word error, interpreted by
MSBuild's cmake as an... error.

Thanks to the internet, at
https://stackoverflow.com/questions/78622876/visual-studio-msbuild-error-msb8066-custom-build
and
https://developercommunity.visualstudio.com/t/MSBuild:-error:-output-of-custom-build/10554390?sort=newest,
and @ccfelius that first raised the problem.

Fixes #16177, also reverts
#16722.

This should unlock #16196 and
#16463.
krlmlr added a commit to duckdb/duckdb-r that referenced this pull request May 15, 2025
@krlmlr
vendor: Update vendored sources to duckdb/duckdb@96c8012
d8b2290 
Implement UUID v7 (duckdb/duckdb#15819)
add workaround for patching httpfs ext (duckdb/duckdb#16722)
krlmlr added a commit to duckdb/duckdb-r that referenced this pull request May 15, 2025
@krlmlr
vendor: Update vendored sources to duckdb/duckdb@96c8012
bbbe81a 
Implement UUID v7 (duckdb/duckdb#15819)
add workaround for patching httpfs ext (duckdb/duckdb#16722)
krlmlr added a commit to duckdb/duckdb-r that referenced this pull request May 16, 2025
@krlmlr
vendor: Update vendored sources to duckdb/duckdb@96c8012
9af5ddf 
Implement UUID v7 (duckdb/duckdb#15819)
add workaround for patching httpfs ext (duckdb/duckdb#16722)
krlmlr added a commit to duckdb/duckdb-r that referenced this pull request May 16, 2025
@krlmlr
vendor: Update vendored sources to duckdb/duckdb@96c8012
27aa269 
Implement UUID v7 (duckdb/duckdb#15819)
add workaround for patching httpfs ext (duckdb/duckdb#16722)
krlmlr added a commit to duckdb/duckdb-r that referenced this pull request May 17, 2025
@krlmlr
vendor: Update vendored sources to duckdb/duckdb@96c8012
90d77fc 
Implement UUID v7 (duckdb/duckdb#15819)
add workaround for patching httpfs ext (duckdb/duckdb#16722)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@carlopi carlopi carlopi approved these changes

Assignees
No one assigned
Labels
Ready To Merge
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@samansmink @Mytherin @carlopi
0