Streaming zlib compressor and decompressor for ReactPHP, supporting compression and decompression of GZIP, ZLIB and raw DEFLATE formats.
Note: This project is in beta stage! Feel free to report any issues you encounter.
Table of contents
Once installed, you can use the following code to pipe a readable gzip file stream into an decompressor which emits decompressed data events for each individual log file chunk:
$loop = React\EventLoop\Factory::create();
$stream = new React\Stream\ReadableResourceStream(fopen('access.log.gz', 'r'), $loop);
$decompressor = new Clue\React\Zlib\Decompressor(ZLIB_ENCODING_GZIP);
$stream->pipe($decompressor);
$decompressor->on('data', function ($data) {
echo $data; // chunk of decompressed log data
});
$loop->run();See also the examples.
This library is a lightweight wrapper around the underlying zlib library. The zlib library offers a number of different formats (sometimes referred to as encodings) detailled below.
This library supports the GZIP compression format as defined in RFC 1952. This is one of the more common compression formats and is used in several places:
- PHP:
ZLIB_ENCODING_GZIP(PHP 5.4+ only) - PHP:
gzdecode()(PHP 5.4+ only) andgzencode() - Files with
.gzfile extension, e.g..tar.gzor.tgzarchives (also known as "tarballs") gzipandgunzip(and family) command line tools- HTTP compression with
Content-Encoding: gzipheader - Java:
GZIPOutputStream
Technically, this format uses raw DEFLATE compression wrapped in a GZIP header and footer:
10 bytes header (+ optional headers) + raw DEFLATE body + 8 bytes footer
This library supports the raw DEFLATE compression format as defined in RFC 1951. The DEFLATE compression algorithm returns what we refer to as "raw DEFLATE format". This raw DEFLATE format is commonly wrapped in container formats instead of being used directly:
- PHP:
ZLIB_ENCODING_RAW(PHP 5.4+ only) - PHP:
gzdeflate()andgzinflate() - Wrapped in GZIP format
- Wrapped in ZLIB format
Note: This format is not to be confused with what some people call "deflate format" or "deflate encoding". These names are commonly used to refer to what we call ZLIB format.
This library supports the ZLIB compression format as defined in RFC 1950. This format is commonly used in a streaming context:
- PHP:
ZLIB_ENCODING_DEFLATE(PHP 5.4+ only) - PHP:
gzcompress()andgzuncompress() - HTTP compression with
Content-Encoding: deflateheader - Java:
DeflaterOutputStream - Qt's
qCompress()andqUncompress()uses the ZLIB format prefixed with the uncompressed length (asUINT32BE).
Technically, this format uses raw DEFLATE compression wrapped in a ZLIB header and footer:
2 bytes header (+ optional headers) + raw DEFLATE body + 4 bytes footer
Note: This format is often referred to as the "deflate format" or "deflate encoding". This documentation avoids this name in order to avoid confusion with the raw DEFLATE format.
All classes use the Clue\React\Zlib namespace.
The Compressor class can be used to compress a stream of data.
It implements the DuplexStreamInterface
and accepts uncompressed data on its writable side and emits compressed data
on its readable side.
$encoding = ZLIB_ENCODING_GZIP; // or ZLIB_ENCODING_RAW or ZLIB_ENCODING_DEFLATE
$compressor = new Clue\React\Zlib\Compressor($encoding);
$compressor->on('data', function ($data) {
echo $data; // compressed binary data chunk
});
$compressor->write($uncompressed); // write uncompressed data chunkThis is particularly useful in a piping context:
$input->pipe($filterBadWords)->pipe($compressor)->pipe($output);For more details, see ReactPHP's
DuplexStreamInterface.
Internally, it implements the deprecated
ZlibFilterStreamclass only for BC reasons. For best forwards compatibility, you should only rely on it implementing theDuplexStreamInterface.
The Decompressor class can be used to decompress a stream of data.
It implements the DuplexStreamInterface
and accepts compressed data on its writable side and emits decompressed data
on its readable side.
$encoding = ZLIB_ENCODING_GZIP; // or ZLIB_ENCODING_RAW or ZLIB_ENCODING_DEFLATE
$decompressor = new Clue\React\Zlib\Decompressor($encoding);
$decompressor->on('data', function ($data) {
echo $data; // decompressed data chunk
});
$decompressor->write($compressed); // write compressed binary data chunkThis is particularly useful in a piping context:
$input->pipe($decompressor)->pipe($filterBadWords)->pipe($output);For more details, see ReactPHP's
DuplexStreamInterface.
Internally, it implements the deprecated
ZlibFilterStreamclass only for BC reasons. For best forwards compatibility, you should only rely on it implementing theDuplexStreamInterface.
The deprecated ZlibFilterStream is a small wrapper around the underlying zlib.deflate and zlib.inflate
stream compression filters offered via ext-zlib.
The following deprecated methods can be used to
create a Compressor instance with the respective encoding parameter:
// deprecated
$compressor = ZlibFilterStream::createGzipCompressor
99F1
();
$compressor = ZlibFilterStream::createDeflateCompressor();
$compressor = ZlibFilterStream::createZlibCompressor();Using any of these methods is deprecated.
Instead, you should explicitly create a Compressor like this:
$encoding = ZLIB_ENCODING_GZIP; // or ZLIB_ENCODING_RAW or ZLIB_ENCODING_DEFLATE
$compressor = new Clue\React\Zlib\Compressor($encoding);See also Compressor for more details.
The following deprecated methods can be used to
create a Decompressor instanceof with the respective encoding parameter:
// deprecated
$decompressor = ZlibFilterStream::createGzipDecompressor();
$decompressor = ZlibFilterStream::createDeflateDecompressor();
$decompressor = ZlibFilterStream::createZlibDecompressor();Using any of these methods is deprecated.
Instead, you should explicitly create a Decompressor like this:
$encoding = ZLIB_ENCODING_GZIP; // or ZLIB_ENCODING_RAW or ZLIB_ENCODING_DEFLATE
$decompressor = new Clue\React\Zlib\Decompressor($encoding);See also Compressor for more details.
The stream compression filters are not exactly the most commonly used features of PHP. As such, we've spotted several inconsistencies (or bugs) between different PHP versions and HHVM. These inconsistencies exist in the underlying PHP engines and there's little we can do about this in this library.
- All Zend PHP versions: Decompressing invalid data does not emit any data (and does not raise an error)
- HHVM only: does not currently support the GZIP and ZLIB format at all (and does not raise an error)
- HHVM only: The
zlib.deflatefilter function buffers the whole string. This means that compressing a stream of 100 MB actually stores the whole string in memory before invoking the underlying compression algorithm. - PHP 5.3 only: Tends to SEGFAULT occasionally on shutdown?
Our test suite contains several test cases that exhibit these issues. If you feel some test case is missing or outdated, we're happy to accept PRs! :)
The recommended way to install this library is through Composer. New to Composer?
While in beta, this project does not currently follow SemVer. This will install the latest supported version:
$ composer require clue/zlib-react:^0.2.2See also the CHANGELOG for details about version upgrades.
This project aims to run on any platform and thus does not require any PHP extensions and supports running on legacy PHP 5.3 through current PHP 7+ and HHVM. It's highly recommended to use PHP 7+ for this project. Older PHP versions may suffer from a number of inconsistencies documented above.
The ext-zlib extension is not required to install this library, however it
is required to actually do anything meaningful with this library.
Each of the above methods will throw an Exception if this extension is
missing.
To run the test suite, you first need to clone this repo and then install all dependencies through Composer:
$ composer installTo run the test suite, go to the project root and run:
$ php vendor/bin/phpunitThis project is released under the permissive MIT license.
Did you know that I offer custom development services and issuing invoices for sponsorships of releases and for contributions? Contact me (@clue) for details.
- If you want to learn more about processing streams of data, refer to the documentation of the underlying react/stream component
- If you want to process compressed tarballs (
.tar.gzand.tgzfile extension), you may want to use clue/reactphp-tar on the decompressed stream.