Description
What happens?
Description
Problem: trying to read a parquet file directly from S3 instance using duckdb
import Pkg
using DuckDB
Pkg.status("DuckDB")
> DuckDB v1.3.0Then try a simple query
conn = DBInterface.connect(DuckDB.DB)
result = DBInterface.execute(conn, "SELECT * FROM 'https://example.com/data.parquet'")
> ERROR: Extension Autoloading Error: An error occurred while trying to automatically install the required extension 'httpfs':
Failed to download extension "httpfs" at URL "http://extensions.duckdb.org/v1.3.0/x86_64-linux-gnu/httpfs.duckdb_extension.gz" (HTTP 403)
Extension "httpfs" is an existing extension.
For more info, visit https://duckdb.org/docs/stable/extensions/troubleshooting/?version=v1.3.0&platform=x86_64-linux-gnu&extension=httpfsNote the x86_64_-linux-gnu version of the extension.
It does turn out that this extension version does not show up on the website on extension distribution and there is no mention of x86_64, though I am guessing in the table it should really be linux_amd64, because there is nothing at the url in the error message.
I followed for a local installation and proceeded with linux_amd64
$ wget http://extensions.duckdb.org/v1.3.0/linux_amd64/httpfs.duckdb_extension.gz
$ gunzip httpfs.duckdb_extension.gzAnd now in julia
using DuckDB
conn = DBInterface.connect(DuckDB.DB)
DBInterface.execute(conn, "INSTALL 'httpfs.duckdb_extension'")
> ERROR: Execute of query "INSTALL 'httpfs.duckdb_extension'" failed: IO Error: Failed to install 'httpfs.duckdb_extension'
The file was built for the platform 'linux_amd64', but we can only load extensions built for platform 'x86_64-linux-gnu'.
Stacktrace:
[1] execute(stmt::DuckDB.Stmt, params::@NamedTuple{})
@ DuckDB /home/eloualic/eloualic/.local/julia/packages/DuckDB/eyW0R/src/result.jl:722
[2] execute
@ /home/eloualic/eloualic/.local/julia/packages/DuckDB/eyW0R/src/result.jl:872 [inlined]
[3] execute
@ /home/eloualic/eloualic/.local/julia/packages/DBInterface/nQcsk/src/DBInterface.jl:130 [inlined]
[4] execute(conn::DuckDB.DB, sql::String)
@ DBInterface /home/eloualic/eloualic/.local/julia/packages/DBInterface/nQcsk/src/DBInterface.jl:152
[5] top-level scope
@ REPL[3]:1Bug being reported
I think this is the bug I'd like to report
- Duckdb cannot install itself the extension because it pulls what looks like a stale url
- Even when I pull the correct url by hand, I still cannot build (the
linux_amd64extensions cant install when duckdb expects ax86_64-linux-gnuextension).
Other information about the system and duckdb on the system
Other piece of information.
I have duckdb cli installed on the same machine (v1.1.1 older version because installed with spack) and httpfs extension did install from inside the cli, so this is why I believe it is a julia issue (or something that happened between 1.1.1 and 1.3.0).
System is a hpc running rocky linux with glibc 2.28
$ uname -sr
> Linux 4.18.0-553.8.1.el8_10.x86_64 Thank you for your help and the amazing work in duckdb.
To Reproduce
Step 1
conn = DBInterface.connect(DuckDB.DB)
result = DBInterface.execute(conn, "SELECT * FROM 'https://example.com/data.parquet'")
> ERROR: Extension Autoloading Error: An error occurred while trying to automatically install the required extension 'httpfs':
Failed to download extension "httpfs" at URL "http://extensions.duckdb.org/v1.3.0/x86_64-linux-gnu/httpfs.duckdb_extension.gz" (HTTP 403)
Extension "httpfs" is an existing extension.
For more info, visit https://duckdb.org/docs/stable/extensions/troubleshooting/?version=v1.3.0&platform=x86_64-linux-gnu&extension=httpfsStep 2
$ wget http://extensions.duckdb.org/v1.3.0/linux_amd64/httpfs.duckdb_extension.gz
$ gunzip httpfs.duckdb_extension.gzAnd now in julia
using DuckDB
conn = DBInterface.connect(DuckDB.DB)
DBInterface.execute(conn, "INSTALL 'httpfs.duckdb_extension'")
> ERROR: Execute of query "INSTALL 'httpfs.duckdb_extension'" failed: IO Error: Failed to install 'httpfs.duckdb_extension'
The file was built for the platform 'linux_amd64', but we can only load extensions built for platform 'x86_64-linux-gnu'.
Stacktrace:
[1] execute(stmt::DuckDB.Stmt, params::@NamedTuple{})
@ DuckDB /home/eloualic/eloualic/.local/julia/packages/DuckDB/eyW0R/src/result.jl:722
[2] execute
@ /home/eloualic/eloualic/.local/julia/packages/DuckDB/eyW0R/src/result.jl:872 [inlined]
[3] execute
@ /home/eloualic/eloualic/.local/julia/packages/DBInterface/nQcsk/src/DBInterface.jl:130 [inlined]
[4] execute(conn::DuckDB.DB, sql::String)
@ DBInterface /home/eloualic/eloualic/.local/julia/packages/DBInterface/nQcsk/src/DBInterface.jl:152
[5] top-level scope
@ REPL[3]:1OS:
Rocky Linux
DuckDB Version:
1.3.0
DuckDB Client:
Julia
Hardware:
No response
Full Name:
Erik Loualiche
Affiliation:
University of Minnesota
What is the latest build you tested with? If possible, we recommend testing with the latest nightly build.
I have not tested with any build
Did you include all relevant data sets for reproducing the issue?
Not applicable - the reproduction does not require a data set
Did you include all code required to reproduce the issue?
- Yes, I have
Did you include all relevant configuration (e.g., CPU architecture, Python version, Linux distribution) to reproduce the issue?
- Yes, I have