8000 Dev by Cyan4973 · Pull Request #3 · facebook/zstd · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

Dev #3

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Jan 25, 2015
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 9 additions & 9 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
**ZSTD**, short for Z-Standard, is a new lossless compression algorithm, which provides both good compression ratio _and_ speed for your standard compression needs. "Standard" translates into everyday situations which neither look for highest possible ratio (which LZMA and ZPAQ cover) nor extreme speeds (which LZ4 covers).
**Zstd**, short for Zstandard, is a new lossless compression algorithm, which provides both good compression ratio _and_ speed for your standard compression needs. "Standard" translates into everyday situations which neither look for highest possible ratio (which LZMA and ZPAQ cover) nor extreme speeds (which LZ4 covers).

It is provided as a BSD-license package, hosted on Github.

Expand All @@ -13,7 +13,7 @@ For a taste of its performance, here are a few benchmark numbers, completed on a
|---------------|-------|---------|---------|
| | | MB/s | MB/s |
| [zlib 1.2.8 -6](http://www.zlib.net/)| 3.099 | 18 | 275 |
| **ZSTD** |**2.872**|**201**|**498** |
| **zstd** |**2.872**|**201**|**498** |
| [zlib 1.2.8 -1](http://www.zlib.net/)| 2.730 | 58 | 250 |
| [LZ4 HC r127](https://github.com/Cyan4973/lz4)| 2.720 | 26 | 1720 |
| QuickLZ 1.5.1b6|2.237 | 323 | 373 |
Expand All @@ -22,24 +22,24 @@ For a taste of its performance, here are a few benchmark numbers, completed on a
| [LZ4 r127](https://github.com/Cyan4973/lz4)| 2.084 | 370 | 1590 |
| LZF 3.6 | 2.077 | 220 | 502 |

An interesting feature of ZSTD is that it can qualify as both a reasonably strong compressor and a fast one.
An interesting feature of zstd is that it can qualify as both a reasonably strong compressor and a fast one.

ZSTD delivers high decompression speed, at around ~500 MB/s per core.
Zstd delivers high decompression speed, at around ~500 MB/s per core.
Obviously, your exact mileage will vary depending on your target system.

ZSTD compression speed, on the other hand, can be configured to fit different situations.
Zstd compression speed, on the other hand, can be configured to fit different situations.
The first, fast, derivative offers ~200 MB/s per core, which is suitable for a few real-time scenarios.
But similar to [LZ4](https://github.com/Cyan4973/lz4), ZSTD can offer derivatives trading compression time for compression ratio, while keeping decompression properties intact. "Offline compression", where compression time is of little importance because the content is only compressed once and decompressed many times, is therefore within the scope.
But similar to [LZ4](https://github.com/Cyan4973/lz4), zstd can offer derivatives trading compression time for compression ratio, while keeping decompression properties intact. "Offline compression", where compression time is of little importance because the content is only compressed once and decompressed many times, is therefore within the scope.

Note that high compression derivatives still have to be developed.
It's a complex area which will certainly benefit the contributions from a few experts.


Another property ZSTD is developed for is configurable memory requirement, with the objective to fit into low-memory configurations, or servers handling many connections in parallel.
Another property zstd is developed for is configurable memory requirement, with the objective to fit into low-memory configurations, or servers handling many connections in parallel.

ZSTD entropy stage is provided by [FSE (Finite State Entropy)](https://github.com/Cyan4973/FiniteStateEntropy).
Zstd entropy stage is provided by [FSE (Finite State Entropy)](https://github.com/Cyan4973/FiniteStateEntropy).

ZSTD development is starting. So consider current results merely as early ones. The implementation will gradually evolve and improve overtime, especially during this first year. This is a phase which will depend a lot on user feedback, since these feedback will be key in deciding next priorities or features to add.
Zstd development is starting. So consider current results merely as early ones. The implementation will gradually evolve and improve overtime, especially during this first year. This is a phase which will depend a lot on user feedback, since these feedback will be key in deciding next priorities or features to add.

The "master" branch is reserved for stable release and betas.
The "dev" branch is the one where all contributions will be merged. If you plan to propose a patch, please commit into the "dev" branch. Direct commit to "master" are not permitted.
Expand Down
4 changes: 2 additions & 2 deletions lib/fse.c
Original file line number Diff line number Diff line change
Expand Up @@ -195,9 +195,9 @@ static void FSE_writeLE64(void* memPtr, U64 val64)
static size_t FSE_readLEST(const void* memPtr)
{
if (sizeof(size_t)==4)
return FSE_readLE32(memPtr);
return (size_t)FSE_readLE32(memPtr);
else
return FSE_readLE64(memPtr);
return (size_t)FSE_readLE64(memPtr);
}

static void FSE_writeLEST(void* memPtr, size_t val)
Expand Down
3 changes: 2 additions & 1 deletion lib/zstd.c
Original file line number Diff line number Diff line change
Expand Up @@ -382,7 +382,8 @@ static unsigned ZSTD_NbCommonBytes (register size_t val)
return (__builtin_clzll(val) >> 3);
# else
unsigned r;
if (!(val>>32)) { r=4; } else { r=0; val>>=32; }
const unsigned n32 = sizeof(size_t)*4; /* calculate this way due to compiler complaining in 32-bits mode */
if (!(val>>n32)) { r=4; } else { r=0; val>>=n32; }
if (!(val>>16)) { r+=2; val>>=8; } else { val>>=24; }
r += (!val);
return r;
Expand Down
0