Boost C++ Libraries: Ticket #5237: filtering_ostream produce incorrect empty files with gzip_compressor and other filters

anonymous — Thu, 10 Nov 2011 01:10:43 GMT

I've noticed this too in Boost 1.44.0.

Davin — Fri, 06 Jan 2012 20:24:41 GMT

Same issue in 1.47.0 and 1.48.0.

You can patch boost/iostreams/filter/gzip.hpp, at line 251, insert the following two lines:

if (!(flags_ & f_header_done))
  write(snk, 0, 0);

If nothing has been written, calling write with no content will generate a proper empty gzip file. This is preferable to having a zero-length file for two reasons. First, it is consistent with the bzip2 filter to create a proper empty compressed file rather than a zero-length file. Second, zcat won't subsequently barf with a "unexpected end of file" error.

Hope it helps.

anonymous — Fri, 24 Aug 2012 10:38:16 GMT

Verified with 1.48.0 as well.

anonymous — Mon, 29 Oct 2012 05:33:04 GMT

Thank you. This works for my situation.

mefyl — Fri, 20 Sep 2013 13:05:15 GMT

This bug has been reported 3 years ago, an apparently working patch has been submitted almost two years ago and yet the bug is still present in Boost 1.53. With all due respect, I expect more from a high standard library such as Boost.

Fri, 20 Sep 2013 13:52:27 GMT

and also present in boost 1.54

anonymous — Tue, 10 Dec 2013 03:44:15 GMT

Still present in 1.55, perhaps someone thinks this is a feature.

anonymous — Wed, 12 Nov 2014 13:38:55 GMT

This is ridiculous. I cannot imagine how this ever got accepted - was there no test suite for the GZip compression/decompression? Surely there are some tests for data round-tripping correctly, and a zero-length stream is an obvious case to include. Or does the decompressor silently ignore broken streams?

cc set

tiwoc@… — Thu, 27 Aug 2015 17:18:31 GMT

cc tiwoc@… added

This is still present in 1.59.0. We now have invalid data in a production database :(

anonymous — Wed, 02 Sep 2015 01:48:34 GMT

Fixed in commit b8db28c0ac42cb40fd78de45ac1217f196e3af4f in branch develop. Tested on msvc 10 and 14, gcc 4.4.7 and 4.9.1, and clang 3.4.2.

I used the suggested fix, calling write() from within close(), except it was necessary to make a small change to write() so that gzip_compressor can be composed with a device or filter that does not support write().

Test case added.

version changed

anonymous — Wed, 02 Sep 2015 01:49:20 GMT

version Boost 1.44.0 → Boost 1.60.0

tiwoc@… — Wed, 02 Sep 2015 07:54:18 GMT

That's great, thank you!

status changed; resolution set

Jonathan Turkanis — Thu, 03 Sep 2015 16:56:50 GMT

status new → closed
resolution → fixed

xiaopeifenng@… — Thu, 23 Feb 2017 09:43:10 GMT

also present in 1.55

anonymous — Thu, 23 Feb 2017 12:09:12 GMT

Replying to xiaopeifenng@…:

also present in 1.55

Please upgrade Boost. This has been fixed in 1.60.0 as written above.