filtering_ostream produce incorrect empty files with gzip_compressor and other filters

[come.raczy@…]

The attached program should produce an empty gzip file, using the gzip_compressor:

int main()
{
    using namespace boost::iostreams;
    filtering_ostream os;
    os.push(boost::iostreams::gzip_compressor());
    os.push(boost::iostreams::file_sink("emptyGzipBug.txt.gz"));
}

The resulting file is not a gzip file:

$ hexdump emptyGzipBug.txt.gz 
0000000 0003                                   
0000002
$ zcat testGzipEmpty.txt.gz 

zcat: testGzipEmpty.txt.gz: not in gzip format
$

A workaround is to manually force an empty write into the stack of components with something like this:

    char dummy = 'b';
    os.component<gzip_compressor>(0)->write(*os.component<file_sink>(1), &dummy, 0);

The problem seems to affect all versions, including Boost 1.46.0

[anonymous]

I've noticed this too in Boost 1.44.0.

[Davin]

Same issue in 1.47.0 and 1.48.0.

You can patch boost/iostreams/filter/gzip.hpp, at line 251, insert the following two lines:

if (!(flags_ & f_header_done))
  write(snk, 0, 0);

If nothing has been written, calling write with no content will generate a proper empty gzip file. This is preferable to having a zero-length file for two reasons. First, it is consistent with the bzip2 filter to create a proper empty compressed file rather than a zero-length file. Second, zcat won't subsequently barf with a "unexpected end of file" error.

Hope it helps.

[anonymous]

Verified with 1.48.0 as well.

[anonymous]

Thank you. This works for my situation.

[mefyl]

This bug has been reported 3 years ago, an apparently working patch has been submitted almost two years ago and yet the bug is still present in Boost 1.53. With all due respect, I expect more from a high standard library such as Boost.

[Come Raczy <come.raczy@…>]

and also present in boost 1.54

[anonymous]

Still present in 1.55, perhaps someone thinks this is a feature.

[anonymous]

This is ridiculous. I cannot imagine how this ever got accepted - was there no test suite for the GZip compression/decompression? Surely there are some tests for data round-tripping correctly, and a zero-length stream is an obvious case to include. Or does the decompressor silently ignore broken streams?

[tiwoc@…]

This is still present in 1.59.0. We now have invalid data in a production database :(

[anonymous]

Fixed in commit ​b8db28c0ac42cb40fd78de45ac1217f196e3af4f in branch develop. Tested on msvc 10 and 14, gcc 4.4.7 and 4.9.1, and clang 3.4.2.

I used the suggested fix, calling write() from within close(), except it was necessary to make a small change to write() so that gzip_compressor can be composed with a device or filter that does not support write().

Test case added.

[tiwoc@…]

That's great, thank you!

[xiaopeifenng@…]

also present in 1.55

[anonymous]

Replying to xiaopeifenng@…:

also present in 1.55

Please upgrade Boost. This has been fixed in 1.60.0 as written above.