8000 Merge V1.2 -> Main by pdet · Pull Request #16751 · duckdb/duckdb · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

Merge V1.2 -> Main #16751

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 42 commits into from
Mar 21, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
42 commits
Select commit Hold shift + click to select a range
482023c
Fix error related to flushing invalid utf-8 values
pdet Mar 12, 2025
1436694
We cant emit the borked rows from casting errors during flushing
pdet Mar 12, 2025
c536b3f
Wip - changing to a linux machine
pdet Mar 13, 2025
f946130
Cleanup this function
pdet Mar 13, 2025
5122d2c
AHA, this is the culprit
pdet Mar 13, 2025
9d258f8
Yaaay
pdet Mar 13, 2025
7093aca
Smaller fixes
pdet Mar 13, 2025
4988b17
tidy
pdet Mar 14, 2025
415f360
Avoid UMA
krlmlr Mar 14, 2025
c965752
Avoid UMA
krlmlr Mar 14, 2025
cdd8c27
Avoid UMA
krlmlr Mar 14, 2025
ec5a670
Tidy again
pdet Mar 14, 2025
2ae23d6
only publish separate libs
taniabogatsch Mar 14, 2025
c6c3a1e
move OrderPreservationRecursive declaration to hpp
jeewonhh Mar 14, 2025
970fed9
format fix
jeewonhh Mar 14, 2025
078e96a
Correctly detect the situation where we have 3 bytes remaining
Mytherin Mar 17, 2025
b9ca61f
Fix #16627: handle DISTINCT FROM and NOT DISTINCT FROM in zone-map pu…
Mytherin Mar 17, 2025
2243dd8
move OrderPreservationRecursive to physical_plan_generator.hpp (#16656)
Mytherin Mar 17, 2025
c8dcf26
Add libs folder to bundled static libs (#16655)
Mytherin Mar 17, 2025
d76de45
Avoid UMA (#16641)
Mytherin Mar 17, 2025
bc4441b
Avoid UMA (#16643)
Mytherin Mar 17, 2025
b537582
Avoid UMA (#16642)
Mytherin Mar 17, 2025
5f6fc8f
Fix #16554: fix blobs as part of .dump
Mytherin Mar 17, 2025
1f7c2c9
Several fixes for CSV fuzzer tests (#16636)
Mytherin Mar 17, 2025
29d2fb6
Issue #16649: SelectNth Remainders
hawkfish Mar 18, 2025
1378dcc
FSST Fix: Correctly detect the situation where we have 3 bytes remain…
Mytherin Mar 18, 2025
745bc78
Fix #16627: handle DISTINCT FROM and NOT DISTINCT FROM in zone-map pu…
Mytherin Mar 18, 2025
48c51dd
Fix #16554: emit blobs as part of .dump (#16693)
Mytherin Mar 18, 2025
459f8df
add avro
samansmink Mar 18, 2025
aa0779f
autoload avro
samansmink Mar 18, 2025
6610bc9
add avro (#16708)
Mytherin Mar 18, 2025
a1a118f
Issue #16652: Implicit Ordered Aggregation
hawkfish Mar 18, 2025
40eac1c
Issue #16652: Implicit Ordered Aggregation (#16718)
Mytherin Mar 19, 2025
7b5e2b7
Issue #16649: SelectNth Remainders (#16705)
Mytherin Mar 19, 2025
9c49af8
fixed the specific issue, added a safeguard against this by just alwa…
Tishj Mar 19, 2025
09bf1d7
[Dev] Fix `EXPORT DATABASE` missing semicolons in produced `schema.sq…
Mytherin Mar 19, 2025
608a9b9
Always throw the error that happens the earliest in the CSV Reader
pdet Mar 19, 2025
0d2f28c
Always throw the error that happens the earliest in the CSV Reader (#…
Mytherin Mar 20, 2025
915aded
Merge remote-tracking branch 'upstream/v1.2-histrionicus' into main_f…
pdet Mar 20, 2025
62f5f5b
Minor tweaks for merge
pdet Mar 20, 2025
2f97b96
Always initialize these bools
pdet Mar 20, 2025
1e9baa7
Remove avro for out_of_tree_extensions.cmake
pdet Mar 21, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
41 changes: 21 additions & 20 deletions .github/workflows/BundleStaticLibs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -80,7 +80,8 @@ jobs:

- name: Bundle static library
shell: bash
run: make bundle-library-o
run: |
make gather-libs

- name: Print platform
shell: bash
Expand All @@ -93,14 +94,14 @@ jobs:
AWS_SECRET_ACCESS_KEY: ${{ secrets.S3_DUCKDB_STAGING_KEY }}
run: |
python3 scripts/amalgamation.py
zip -j static-lib-osx-${{ matrix.architecture }}.zip src/include/duckdb.h build/release/libduckdb_bundle.a
./scripts/upload-assets-to-staging.sh github_release static-lib-osx-${{ matrix.architecture }}.zip
zip -r -j static-libs-osx-${{ matrix.architecture }}.zip src/include/duckdb.h build/release/libs/
./scripts/upload-assets-to-staging.sh github_release static-libs-osx-${{ matrix.architecture }}.zip

- uses: actions/upload-artifact@v4
with:
name: duckdb-static-lib-osx-${{ matrix.architecture }}
name: duckdb-static-libs-osx-${{ matrix.architecture }}
path: |
static-lib-osx-${{ matrix.architecture }}.zip
static-libs-osx-${{ matrix.architecture }}.zip

bundle-mingw-static-lib:
name: Windows MingW static libs
Expand Down Expand Up @@ -142,22 +143,22 @@ jobs:
- name: Bundle static library
shell: bash
run: |
make bundle-library-obj
make gather-libs

- name: Deploy
shell: bash
env:
AWS_ACCESS_KEY_ID: ${{ secrets.S3_DUCKDB_STAGING_ID }}
AWS_SECRET_ACCESS_KEY: ${{ secrets.S3_DUCKDB_STAGING_KEY }}
run: |
zip -j static-lib-windows-mingw.zip src/include/duckdb.h build/release/libduckdb_bundle.a
./scripts/upload-assets-to-staging.sh github_release static-lib-windows-mingw.zip
zip -r -j static-libs-windows-mingw.zip src/include/duckdb.h build/release/libs/
./scripts/upload-assets-to-staging.sh github_release static-libs-windows-mingw.zip

- uses: actions/upload-artifact@v4
with:
name: duckdb-static-lib-windows-mingw
name: duckdb-static-libs-windows-mingw
path: |
static-lib-windows-mingw.zip
static-libs-windows-mingw.zip
bundle-linux-arm64-static-libs:
name: Linux arm64 static libs
runs-on: ubuntu-latest
Expand All @@ -183,21 +184,21 @@ jobs:
-e FORCE_WARN_UNUSED=1 \
-e DUCKDB_PLATFORM=linux_arm64 \
ubuntu:18.04 \
bash -c "/duckdb/scripts/setup_ubuntu1804.sh && git config --global --add safe.directory /duckdb && make bundle-library -C /duckdb"
bash -c "/duckdb/scripts/setup_ubuntu1804.sh && git config --global --add safe.directory /duckdb && make gather-libs -C /duckdb"
- name: Deploy
shell: bash
env:
AWS_ACCESS_KEY_ID: ${{ secrets.S3_DUCKDB_STAGING_ID }}
AWS_SECRET_ACCESS_KEY: ${{ secrets.S3_DUCKDB_STAGING_KEY }}
run: |
python3 scripts/amalgamation.py
zip -j static-lib-linux-arm64.zip src/include/duckdb.h build/release/libduckdb_bundle.a
./scripts/upload-assets-to-staging.sh github_release static-lib-linux-arm64.zip
zip -r -j static-libs-linux-arm64.zip src/include/duckdb.h build/release/libs/
./scripts/upload-assets-to-staging.sh github_release static-libs-linux-arm64.zip
- uses: actions/upload-artifact@v4
with:
name: duckdb-static-lib-linux-arm64
name: duckdb-static-libs-linux-arm64
path: |
static-lib-linux-arm64.zip
static-libs-linux-arm64.zip
bundle-linux-amd64-static-libs:
name: Linux amd64 static libs
runs-on: ubuntu-latest
Expand Down Expand Up @@ -225,7 +226,7 @@ jobs:
-e BUILD_BENCHMARK=1 \
-e FORCE_WARN_UNUSED=1 \
quay.io/pypa/manylinux2014_x86_64 \
bash -c "yum install -y perl-IPC-Cmd && git config --global --add safe.directory $PWD && make bundle-library -C $PWD"
bash -c "yum install -y perl-IPC-Cmd && git config --global --add safe.directory $PWD && make gather-libs -C $PWD"
- name: Print platform
shell: bash
run: ./build/release/duckdb -c "PRAGMA platform;"
Expand All @@ -237,10 +238,10 @@ jobs:
AWS_SECRET_ACCESS_KEY: ${{ secrets.S3_DUCKDB_STAGING_KEY }}
run: |
python3 scripts/amalgamation.py
zip -j static-lib-linux-amd64.zip src/include/duckdb.h build/release/libduckdb_bundle.a
./scripts/upload-assets-to-staging.sh github_release static-lib-linux-amd64.zip
zip -r -j static-libs-linux-amd64.zip src/include/duckdb.h build/release/libs/
./scripts/upload-assets-to-staging.sh github_release static-libs-linux-amd64.zip
- uses: actions/upload-artifact@v4
with:
name: duckdb-static-lib-linux-amd64
name: duckdb-static-libs-linux-amd64
path: |
static-lib-linux-amd64.zip
static-libs-linux-amd64.zip
8 changes: 8 additions & 0 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -524,3 +524,11 @@ bundle-library-obj: bundle-setup

bundle-library: release
make bundle-library-o

gather-libs: release
cd build/release && \
rm -rf libs && \
mkdir -p libs && \
cp src/libduckdb_static.a libs/. && \
cp third_party/*/libduckdb_*.a libs/. && \
cp extension/*/lib*_extension.a libs/.
2 changes: 2 additions & 0 deletions data/csv/afl/4086/case_1.csv
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
Ë
Ì
149 changes: 149 additions & 0 deletions data/csv/afl/4086/case_2.csv
Original file line number Diff line number Diff line change
@@ -0,0 +1,149 @@

>
?
@
A
B
C
D
E
F
G
H
I
J
K
L
M
N
O
P
Q
R
S
T
U
V
W
X
Y
Z
[
\
]
^
_
`
a
b
c
j
k
l

Ë
Ì

F
G
H
I
J
K
L
M
N
O
Í
Î
Ï
Ð
Ñ
Ò
Ó
Ôw
x
y
z
{
|
}
~
Ÿ
¡
¢
£
¤
¥
¦
§
¨
©
ª
«
¬
­
®
¯
°
±
²
³
´
µ
·
¸
¹
º
»
¼
½
¾
¿
À
Á
Â
Ã
Ä
Å
ÿ}}}}}}}}}}}}}}}}}}}}}}
Ç
È
É
Ê
Ë
Ì
Í
Î
Ï
Ð
Ñ
Ò
Ó
Ô
2
3
4
5
Õ
Ö
×
Ø
Ù
Ú
Û
Ü
Ý
Þ
ß
à
á
â
ã
ä
å
æ
ç
è
é
Binary file added data/csv/afl/4086/case_3.csv
Binary file not shown.
Original file line number Diff line number Diff line change
Expand Up @@ -8,17 +8,13 @@

#pragma once

#include "duckdb/common/sort/sort.hpp"
#include "duckdb/common/types/column/column_data_collection.hpp"
#include "duckdb/common/types/row/row_layout.hpp"
#include "core_functions/aggregate/quantile_helpers.hpp"
#include "duckdb/execution/merge_sort_tree.hpp"
#include "duckdb/common/operator/cast_operators.hpp"
#include "duckdb/common/operator/multiply.hpp"
#include "duckdb/planner/expression/bound_constant_expression.hpp"
#include "duckdb/function/window/window_index_tree.hpp"
#include <algorithm>
#include <numeric>
#include <stdlib.h>
#include <utility>

Expand Down Expand Up @@ -89,7 +85,7 @@ struct QuantileDirect {
using RESULT_TYPE = T;

inline const INPUT_TYPE &operator()(const INPUT_TYPE &x) const {
return x;
return x; // NOLINT
}
};

Expand Down Expand Up @@ -365,7 +361,7 @@ struct QuantileSortTree {
}

inline idx_t SelectNth(const SubFrames &frames, size_t n) const {
return index_tree->SelectNth(frames, n);
return index_tree->SelectNth(frames, n).first;
}

template <typename INPUT_TYPE, typename RESULT_TYPE, bool DISCRETE>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -207,6 +207,9 @@ struct WindowQuantileState {
dest[0] = skips[0].second;
if (skips.size() > 1) {
dest[1] = skips[1].second;
} else {
// Avoid UMA
dest[1] = skips[0].second;
}
return interp.template Extract<INPUT_TYPE, RESULT_TYPE>(dest.data(), result);
} catch (const duckdb_skiplistlib::skip_list::IndexError &idx_err) {
Expand Down
1 change: 1 addition & 0 deletions scripts/generate_extensions_function.py
Original file line number Diff line number Diff line change
Expand Up @@ -742,6 +742,7 @@ def write_header(data: ExtensionData):
}; // EXTENSION_SECRET_PROVIDERS

static constexpr const char *AUTOLOADABLE_EXTENSIONS[] = {
"avro",
"aws",
"azure",
"autocomplete",
Expand Down
20 changes: 10 additions & 10 deletions src/common/types/row/tuple_data_segment.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -15,23 +15,23 @@ void TupleDataChunkPart::SetHeapEmpty() {
base_heap_ptr = nullptr;
}

void SwapTupleDataChunkPart(TupleDataChunkPart &a, TupleDataChunkPart &b) {
std::swap(a.row_block_index, b.row_block_index);
std::swap(a.row_block_offset, b.row_block_offset);
std::swap(a.heap_block_index, b.heap_block_index);
std::swap(a.heap_block_offset, b.heap_block_offset);
std::swap(a.base_heap_ptr, b.base_heap_ptr);
std::swap(a.total_heap_size, b.total_heap_size);
std::swap(a.count, b.count);
void MoveTupleDataChunkPart(TupleDataChunkPart &a, TupleDataChunkPart &b) {
a.row_block_index = b.row_block_index;
a.row_block_offset = b.row_block_offset;
a.heap_block_index = b.heap_block_index;
a.heap_block_offset = b.heap_block_offset;
a.base_heap_ptr = b.base_heap_ptr;
a.total_heap_size = b.total_heap_size;
a.count = b.count;
std::swap(a.lock, b.lock);
}

TupleDataChunkPart::TupleDataChunkPart(TupleDataChunkPart &&other) noexcept : lock((other.lock)) {
SwapTupleDataChunkPart(*this, other);
MoveTupleDataChunkPart(*this, other);
}

TupleDataChunkPart &TupleDataChunkPart::operator=(TupleDataChunkPart &&other) noexcept {
SwapTupleDataChunkPart(*this, other);
MoveTupleDataChunkPart(*this, other);
return *this;
}

Expand Down
Loading
Loading
0