8000 Merge v1.4.2 to Master by felixhandte · Pull Request #1700 · facebook/zstd · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

Merge v1.4.2 to Master #1700

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 27 commits into from
Jul 25, 2019
Merged

Merge v1.4.2 to Master #1700

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
27 commits
Select commit Hold shift + click to select a range
de61b36
Merge branch 'master' of https://github.com/vivekmig/zstd into dev
vivekmig Jul 15, 2019
3f108f8
Return error if block size exceeds maximum
vivekmig Jul 15, 2019
1dc98de
Restructure the source files
ephiepark Jul 3, 2019
c7be7d2
Fixing compressed block size checks
vivekmig Jul 17, 2019
a3ce0c9
Fixing decodecorpus test issue
vivekmig Jul 18, 2019
04a9d6b
fix compiling errors with clang-8
Jul 18, 2019
f262069
Merge pull request #1685 from vivekmig/dev
Cyan4973 Jul 19, 2019
be3d2e2
Merge pull request #1679 from ephiepark/dev
Cyan4973 Jul 19, 2019
671d533
Fix seekable decompression in-memory api
sean-purcell Jul 22, 2019
e6edcfa
[legacy] Fix bug in zstd-0.5 decoder
terrelln Jul 22, 2019
740b321
Merge pull request #1696 from terrelln/legacy-fix
terrelln Jul 22, 2019
b0a5d38
Merge pull request #1695 from iburinoc/seekable-buff
Cyan4973 Jul 22, 2019
02da449
Adding documentation for shrink flag PR #1656
Tyler-Tran Jul 22, 2019
f8c1d79
modifying minor nit
Tyler-Tran Jul 22, 2019
12d60a9
removing changes to zstd.1
Tyler-Tran Jul 24, 2019
5a61e66
previous commit did not undo all changes
Tyler-Tra 8000 n Jul 24, 2019
8083581
Bump Library Version Number to 1.4.2
felixhandte Jul 24, 2019
25824cc
Update CHANGELOG
felixhandte Jul 24, 2019
9cb9b1c
Update Manual
felixhandte Jul 24, 2019
15da578
Add New Seekable Compression Example to .gitignore
felixhandte Jul 24, 2019
ff8b18a
Merge pull request #1697 from Tyler-Tran/dev
Cyan4973 Jul 24, 2019
0d38ee3
Merge pull request #1690 from piguin/dev
Cyan4973 Jul 24, 2019
96a24e1
Merge pull request #1698 from felixhandte/bump-version-to-1.4.2
Cyan4973 Jul 24, 2019
1f16245
updated man page
Cyan4973 Jul 24, 2019
2314906
Merge pull request #1699 from felixhandte/seekable-gitignore
felixhandte Jul 24, 2019
793b94b
memory leak fix
LeeYoung624 Jul 25, 2019
d95ac34
Merge pull request #1701 from LeeYoung624/dev
felixhandte Jul 25, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions CHANGELOG
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,3 +1,9 @@
v1.4.2
bug: Fix bug in zstd-0.5 decoder by @terrelln (#1696)
bug: Fix seekable decompression in-memory API by @iburinoc (#1695)
misc: Validate blocks are smaller than size limit by @vivekmg (#1685)
misc: Restructure source files by @ephiepark (#1679)

v1.4.1
bug: Fix data corruption in niche use cases by @terrelln (#1659)
bug: Fuzz legacy modes, fix uncovered bugs by @terrelln (#1593, #1594, #1595)
Expand Down
16 changes: 16 additions & 0 deletions build/VS2008/fullbench/fullbench.vcproj
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -364,6 +364,14 @@
RelativePath="..\..\..\lib\compress\zstd_compress.c"
>
</File>
<File
RelativePath="..\..\..\lib\compress\zstd_compress_literals.c"
>
</File>
<File
RelativePath="..\..\..\lib\compress\zstd_compress_sequences.c"
>
</File>
<File
RelativePath="..\..\..\lib\compress\zstdmt_compress.c"
>
Expand Down Expand Up @@ -494,6 +502,14 @@
RelativePath="..\..\..\lib\compress\zstd_compress.h"
>
</File>
<File
RelativePath="..\..\..\lib\compress\zstd_compress_literals.h"
>
</File>
<File
RelativePath="..\..\..\lib\compress\zstd_compress_sequences.h"
>
</File>
<File
RelativePath="..\..\..\lib\compress\zstd_fast.h"
>
Expand Down
16 changes: 16 additions & 0 deletions build/VS2008/fuzzer/fuzzer.vcproj
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -412,6 +412,14 @@
RelativePath="..\..\..\lib\compress\zstd_compress.c"
>
</File>
<File
RelativePath="..\..\..\lib\compress\zstd_compress_literals.c"
>
</File>
<File
RelativePath="..\..\..\lib\compress\zstd_compress_sequences.c"
>
</File>
<File
RelativePath="..\..\..\lib\decompress\zstd_decompress.c"
>
Expand Down Expand Up @@ -530,6 +538,14 @@
RelativePath="..\..\..\lib\compress\zstd_compress.h"
>
</File>
<File
RelativePath="..\..\..\lib\compress\zstd_compress_literals.h"
>
</File>
<File
RelativePath="..\..\..\lib\compress\zstd_compress_sequences.h"
>
</File>
<File
RelativePath="..\..\..\lib\compress\zstd_fast.h"
>
Expand Down
16 changes: 16 additions & 0 deletions build/VS2008/zstd/zstd.vcproj
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -424,6 +424,14 @@
RelativePath="..\..\..\lib\compress\zstd_compress.c"
>
</File>
<File
RelativePath="..\..\..\lib\compress\zstd_compress_literals.c"
>
</File>
<File
RelativePath="..\..\..\lib\compress\zstd_compress_sequences.c"
>
</File>
<File
RelativePath="..\..\..\lib\decompress\zstd_decompress.c"
>
Expand Down Expand Up @@ -610,6 +618,14 @@
RelativePath="..\..\..\lib\compress\zstd_compress.h"
>
</File>
<File
RelativePath="..\..\..\lib\compress\zstd_compress_literals.h"
>
</File>
<File
RelativePath="..\..\..\lib\compress\zstd_compress_sequences.h"
>
</File>
<File
RelativePath="..\..\..\lib\compress\zstd_fast.h"
>
Expand Down
16 changes: 16 additions & 0 deletions build/VS2008/zstdlib/zstdlib.vcproj
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -396,6 +396,14 @@
RelativePath="..\..\..\lib\compress\zstd_compress.c"
>
</File>
<File
RelativePath="..\..\..\lib\compress\zstd_compress_literals.c"
>
</File>
<File
RelativePath="..\..\..\lib\compress\zstd_compress_sequences.c"
>
</File>
<File
RelativePath="..\..\..\lib\compress\zstd_fast.c"
>
Expand Down Expand Up @@ -542,6 +550,14 @@
RelativePath="..\..\..\lib\compress\zstd_compress.h"
>
</File>
<File
RelativePath="..\..\..\lib\compress\zstd_compress_literals.h"
>
</File>
<File
RelativePath="..\..\..\lib\compress\zstd_compress_sequences.h"
>
</File>
<File
RelativePath="..\..\..\lib\compress\zstd_fast.h"
>
Expand Down
4 changes: 4 additions & 0 deletions build/VS2010/fullbench/fullbench.vcxproj
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -167,6 +167,8 @@
<ClCompile Include="..\..\..\lib\compress\fse_compress.c" />
<ClCompile Include="..\..\..\lib\compress\huf_compress.c" />
<ClCompile Include="..\..\..\lib\compress\zstd_compress.c" />
<ClCompile Include="..\..\..\lib\compress\zstd_compress_literals.c" />
<ClCompile Include="..\..\..\lib\compress\zstd_compress_sequences.c" />
<ClCompile Include="..\..\..\lib\compress\zstdmt_compress.c" />
<ClCompile Include="..\..\..\lib\compress\zstd_fast.c" />
<ClCompile Include="..\..\..\lib\compress\zstd_double_fast.c" />
Expand All @@ -193,6 +195,8 @@
<ClInclude Include="..\..\..\lib\common\threading.h" />
<ClInclude Include="..\..\..\lib\common\xxhash.h" />
<ClInclude Include="..\..\..\lib\compress\zstd_compress.h" />
<ClInclude Include="..\..\..\lib\compress\zstd_compress_literals.h" />
<ClInclude Include="..\..\..\lib\compress\zstd_compress_sequences.h" />
<ClInclude Include="..\..\..\lib\compress\zstd_fast.h" />
<ClInclude Include="..\..\..\lib\compress\zstd_double_fast.h" />
<ClInclude Include="..\..\..\lib\compress\zstd_lazy.h" />
Expand Down
4 changes: 4 additions & 0 deletions build/VS2010/fuzzer/fuzzer.vcxproj
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -167,6 +167,8 @@
<ClCompile Include="..\..\..\lib\compress\fse_compress.c" />
<ClCompile Include="..\..\..\lib\compress\huf_compress.c" />
<ClCompile Include="..\..\..\lib\compress\zstd_compress.c" />
<ClCompile Include="..\..\..\lib\compress\zstd_compress_literals.c" />
<ClCompile Include="..\..\..\lib\compress\zstd_compress_sequences.c" />
<ClCompile Include="..\..\..\lib\compress\zstd_fast.c" />
<ClCompile Include="..\..\..\lib\compress\zstd_double_fast.c" />
<ClCompile Include="..\..\..\lib\compress\zstd_lazy.c" />
Expand Down Expand Up @@ -196,6 +198,8 @@
<ClInclude Include="..\..\..\lib\common\zstd_errors.h" />
<ClInclude Include="..\..\..\lib\zstd.h" />
<ClInclude Include="..\..\..\lib\compress\zstd_compress.h" />
<ClInclude Include="..\..\..\lib\compress\zstd_compress_literals.h" />
<ClInclude Include="..\..\..\lib\compress\zstd_compress_sequences.h" />
<ClInclude Include="..\..\..\lib\compress\zstd_fast.h" />
<ClInclude Include="..\..\..\lib\compress\zstd_double_fast.h" />
<ClInclude Include="..\..\..\lib\compress\zstd_lazy.h" />
Expand Down
4 changes: 4 additions & 0 deletions build/VS2010/libzstd-dll/libzstd-dll.vcxproj
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,8 @@
<ClCompile Include="..\..\..\lib\compress\fse_compress.c" />
<ClCompile Include="..\..\..\lib\compress\huf_compress.c" />
<ClCompile Include="..\..\..\lib\compress\zstd_compress.c" />
<ClCompile Include="..\..\..\lib\compress\zstd_compress_literals.c" />
<ClCompile Include="..\..\..\lib\compress\zstd_compress_sequences.c" />
<ClCompile Include="..\..\..\lib\compress\zstd_fast.c" />
<ClCompile Include="..\..\..\lib\compress\zstd_double_fast.c" />
<ClCompile Include="..\..\..\lib\compress\zstd_lazy.c" />
Expand Down Expand Up @@ -78,6 +80,8 @@
<ClInclude Include="..\..\..\lib\zstd.h" />
<ClInclude Include="..\..\..\lib\common\zstd_internal.h" />
<ClInclude Include="..\..\..\lib\compress\zstd_compress.h" />
<ClInclude Include="..\..\..\lib\compress\zstd_compress_literals.h" />
<ClInclude Include="..\..\..\lib\compress\zstd_compress_sequences.h" />
<ClInclude Include="..\..\..\lib\compress\zstd_fast.h" />
<ClInclude Include="..\..\..\lib\compress\zstd_double_fast.h" />
<ClInclude Include="..\..\..\lib\compress\zstd_lazy.h" />
Expand Down
4 changes: 4 additions & 0 deletions build/VS2010/libzstd/libzstd.vcxproj
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,8 @@
<ClCompile Include="..\..\..\lib\compress\fse_compress.c" />
<ClCompile Include="..\..\..\lib\compress\huf_compress.c" />
<ClCompile Include="..\..\..\lib\compress\zstd_compress.c" />
<ClCompile Include="..\..\..\lib\compress\zstd_compress_literals.c" />
<ClCompile Include="..\..\..\lib\compress\zstd_compress_sequences.c" />
<ClCompile Include="..\..\..\lib\compress\zstd_fast.c" />
<ClCompile Include="..\..\..\lib\compress\zstd_double_fast.c" />
<ClCompile Include="..\..\..\lib\compress\zstd_lazy.c" />
Expand Down Expand Up @@ -78,6 +80,8 @@
<ClInclude Include="..\..\..\lib\zstd.h" />
<ClInclude Include="..\..\..\lib\common\zstd_internal.h" />
<ClInclude Include="..\..\..\lib\compress\zstd_compress.h" />
<ClInclude Include="..\..\..\lib\compress\zstd_compress_literals.h" />
<ClInclude Include="..\..\..\lib\compress\zstd_compress_sequences.h" />
<ClInclude Include="..\..\..\lib\compress\zstd_fast.h" />
<ClInclude Include="..\..\..\lib\compress\zstd_double_fast.h" />
<ClInclude Include="..\..\..\lib\compress\zstd_lazy.h" />
Expand Down
4 changes: 4 additions & 0 deletions build/VS2010/zstd/zstd.vcxproj
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,8 @@
<ClCompile Include="..\..\..\lib\compress\huf_compress.c" />
<ClCompile Include="..\..\..\lib\compress\zstdmt_compress.c" />
<ClCompile Include="..\..\..\lib\compress\zstd_compress.c" />
<ClCompile Include="..\..\..\lib\compress\zstd_compress_literals.c" />
<ClCompile Include="..\..\..\lib\compress\zstd_compress_sequences.c" />
<ClCompile Include="..\..\..\lib\compress\zstd_fast.c" />
<ClCompile Include="..\..\..\lib\compress\zstd_double_fast.c" />
<ClCompile Include="..\..\..\lib\compress\zstd_lazy.c" />
Expand Down Expand Up @@ -75,6 +77,8 @@
<ClInclude Include="..\..\..\lib\common\zstd_internal.h" />
<ClInclude Include="..\..\..\lib\common\zstd_errors.h" />
<ClInclude Include="..\..\..\lib\compress\zstd_compress.h" />
<ClInclude Include="..\..\..\lib\compress\zstd_compress_literals.h" />
<ClInclude Include="..\..\..\lib\compress\zstd_compress_sequences.h" />
<ClInclude Include="..\..\..\lib\compress\zstd_fast.h" />
<ClInclude Include="..\..\..\lib\compress\zstd_double_fast.h" />
<ClInclude Include="..\..\..\lib\compress\zstd_lazy.h" />
Expand Down
74 changes: 22 additions & 52 deletions build/cmake/lib/CMakeLists.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,62 +20,32 @@ endif()
# Define library directory, where sources and header files are located
include_directories(${LIBRARY_DIR} ${LIBRARY_DIR}/common)

file(GLOB CommonSources ${LIBRARY_DIR}/common/*.c)
file(GLOB CompressSources ${LIBRARY_DIR}/compress/*.c)
file(GLOB DecompressSources ${LIBRARY_DIR}/decompress/*.c)
file(GLOB DictBuilderSources ${LIBRARY_DIR}/dictBuilder/*.c)
file(GLOB DeprecatedSources ${LIBRARY_DIR}/deprecated/*.c)

set(Sources
${LIBRARY_DIR}/common/entropy_common.c
${LIBRARY_DIR}/common/fse_decompress.c
${LIBRARY_DIR}/common/threading.c
${LIBRARY_DIR}/common/pool.c
${LIBRARY_DIR}/common/zstd_common.c
${LIBRARY_DIR}/common/error_private.c
${LIBRARY_DIR}/common/xxhash.c
${LIBRARY_DIR}/compress/hist.c
${LIBRARY_DIR}/compress/fse_compress.c
${LIBRARY_DIR}/compress/huf_compress.c
${LIBRARY_DIR}/compress/zstd_compress.c
${LIBRARY_DIR}/compress/zstdmt_compress.c
${LIBRARY_DIR}/compress/zstd_fast.c
${LIBRARY_DIR}/compress/zstd_double_fast.c
${LIBRARY_DIR}/compress/zstd_lazy.c
${LIBRARY_DIR}/compress/zstd_opt.c
${LIBRARY_DIR}/compress/zstd_ldm.c
${LIBRARY_DIR}/decompress/huf_decompress.c
${LIBRARY_DIR}/decompress/zstd_decompress.c
${LIBRARY_DIR}/decompress/zstd_decompress_block.c
${LIBRARY_DIR}/decompress/zstd_ddict.c
${LIBRARY_DIR}/dictBuilder/cover.c
${LIBRARY_DIR}/dictBuilder/fastcover.c
${LIBRARY_DIR}/dictBuilder/divsufsort.c
${LIBRARY_DIR}/dictBuilder/zdict.c
${LIBRARY_DIR}/deprecated/zbuff_common.c
${LIBRARY_DIR}/deprecated/zbuff_compress.c
${LIBRARY_DIR}/deprecated/zbuff_decompress.c)
${CommonSources}
${CompressSources}
${DecompressSources}
${DictBuilderSources}
${DeprecatedSources})

file(GLOB CommonHeaders ${LIBRARY_DIR}/common/*.h)
file(GLOB CompressHeaders ${LIBRARY_DIR}/compress/*.h)
file(GLOB DecompressHeaders ${LIBRARY_DIR}/decompress/*.h)
file(GLOB DictBuilderHeaders ${LIBRARY_DIR}/dictBuilder/*.h)
file(GLOB DeprecatedHeaders ${LIBRARY_DIR}/deprecated/*.h)

set(Headers
${LIBRARY_DIR}/zstd.h
${LIBRARY_DIR}/common/debug.h
${LIBRARY_DIR}/common/pool.h
${LIBRARY_DIR}/common/threading.h
${LIBRARY_DIR}/common/bitstream.h
${LIBRARY_DIR}/common/error_private.h
${LIBRARY_DIR}/common/zstd_errors.h
${LIBRARY_DIR}/common/fse.h
${LIBRARY_DIR}/common/huf.h
${LIBRARY_DIR}/common/mem.h
${LIBRARY_DIR}/common/zstd_internal.h
${LIBRARY_DIR}/compress/hist.h
${LIBRARY_DIR}/compress/zstd_compress_internal.h
${LIBRARY_DIR}/compress/zstd_fast.h
${LIBRARY_DIR}/compress/zstd_double_fast.h
${LIBRARY_DIR}/compress/zstd_lazy.h
${LIBRARY_DIR}/compress/zstd_opt.h
${LIBRARY_DIR}/compress/zstd_ldm.h
${LIBRARY_DIR}/compress/zstdmt_compress.h
${LIBRARY_DIR}/decompress/zstd_decompress_internal.h
${LIBRARY_DIR}/decompress/zstd_decompress_block.h
${LIBRARY_DIR}/decompress/zstd_ddict.h
${LIBRARY_DIR}/dictBuilder/zdict.h
${LIBRARY_DIR}/dictBuilder/cover.h
${LIBRARY_DIR}/deprecated/zbuff.h)
${CommonHeaders}
${CompressHeaders}
${DecompressHeaders}
${DictBuilderHeaders}
${DeprecatedHeaders})

if (ZSTD_LEGACY_SUPPORT)
set(LIBRARY_LEGACY_DIR ${LIBRARY_DIR}/legacy)
Expand Down
2 changes: 2 additions & 0 deletions build/meson/lib/meson.build
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,8 @@ libzstd_sources = [join_paths(zstd_rootdir, 'lib/common/entropy_common.c'),
join_paths(zstd_rootdir, 'lib/compress/fse_compress.c'),
join_paths(zstd_rootdir, 'lib/compress/huf_compress.c'),
join_paths(zstd_rootdir, 'lib/compress/zstd_compress.c'),
join_paths(zstd_rootdir, 'lib/compress/zstd_compress_literals.c'),
join_paths(zstd_rootdir, 'lib/compress/zstd_compress_sequences.c'),
join_paths(zstd_rootdir, 'lib/compress/zstdmt_compress.c'),
join_paths(zstd_rootdir, 'lib/compress/zstd_fast.c'),
join_paths(zstd_rootdir, 'lib/compress/zstd_double_fast.c'),
Expand Down
2 changes: 1 addition & 1 deletion contrib/largeNbDicts/largeNbDicts.c
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -559,7 +559,7 @@ static int benchMem(slice_collection_t dstBlocks,
CONTROL(BMK_isSuccessful_runOutcome(outcome));

BMK_runTime_t const result = BMK_extract_runTime(outcome);
U64 const dTime_ns = result.nanoSecPerRun;
double const dTime_ns = result.nanoSecPerRun;
double const dTime_sec = (double)dTime_ns / 1000000000;
size_t const srcSize = result.sumOfReturn;
double const dSpeed_MBps = (double)srcSize / dTime_sec / (1 MB);
Expand Down
1 change: 1 addition & 0 deletions contrib/seekable_format/examples/.gitignore
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
seekable_compression
seekable_decompression
seekable_decompression_mem
parallel_processing
parallel_compression
7 changes: 6 additions & 1 deletion contrib/seekable_format/examples/Makefile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,8 @@ SEEKABLE_OBJS = ../zstdseek_compress.c ../zstdseek_decompress.c $(ZSTDLIB)

default: all

all: seekable_compression seekable_decompression parallel_processing
all: seekable_compression seekable_decompression seekable_decompression_mem \
parallel_processing

$(ZSTDLIB):
make -C $(ZSTDLIB_PATH) $(ZSTDLIB_NAME)
Expand All @@ -35,6 +36,9 @@ seekable_compression : seekable_compression.c $(SEEKABLE_OBJS)
seekable_decompression : seekable_decompression.c $(SEEKABLE_OBJS)
$(CC) $(CPPFLAGS) $(CFLAGS) $^ $(LDFLAGS) -o $@

seekable_decompression_mem : seekable_decompression_mem.c $(SEEKABLE_OBJS)
$(CC) $(CPPFLAGS) $(CFLAGS) $^ $(LDFLAGS) -o $@

parallel_processing : parallel_processing.c $(SEEKABLE_OBJS)
$(CC) $(CPPFLAGS) $(CFLAGS) $^ $(LDFLAGS) -o $@ -pthread

Expand All @@ -44,5 +48,6 @@ parallel_compression : parallel_compression.c $(SEEKABLE_OBJS)
clean:
@rm -f core *.o tmp* result* *.zst \
seekable_compression seekable_decompression \
seekable_decompression_mem \
parallel_processing parallel_compression
@echo Cleaning completed
Loading
0