Boost C++ Libraries: Ticket #2318: A particular string for which compressing/uncompressing using iostreams/zlib fails.

attachment set

guy_tannenbaum@… — Thu, 11 Sep 2008 07:12:55 GMT

attachment → strange_zlib.cpp

program which generates the behavior

attachment set

guy_tannenbaum@… — Thu, 11 Sep 2008 07:14:05 GMT

attachment → short_magic.txt

problematic string (each line is the ascii code of one char)

guy_tannenbaum@… — Thu, 11 Sep 2008 07:22:09 GMT

Found this on Boost 1.35.0 (not 1.36.0)

rbock@… — Tue, 14 Oct 2008 22:37:54 GMT

Hi,

I found the same problem in 1.36. It is rather rare: In 6.5Million compressed files in my project, it occurred only 441 times. As mentioned by Guy, the size of the damaged zip is a bit peculiar, all sizes I checked are multiples of 4096.

Also, I think it is more related to filtering_istream and filtering_istreambuf than zlib_compression, as it works fine if you use filtering_ostream or filtering_ostreambuf instead.

I am going to attach an extended example with similar code which does not work, too for the given example string and code which does (using filtering_ostream instead of filtering_istream for compression).

I am also attaching a binary version of the sample data, since reading the ascii version did not work for me (I had to replace the lexical_cast by atoi to make it work).

Here is some system information, if this should be required to reproduce the problem (and circumvention):

Ubuntu 8.04 64bit, gcc 4.2.3, boost-1.36, zlib-1g 1.2.3.3

Please do not hesitate to ask for more information. Personally, I consider this a rather severe problem since it can impair data integrity in backups. Lucky me, my software is not in production yet :-)

Regards,

Roland

attachment set

rbock@… — Tue, 14 Oct 2008 22:40:35 GMT

attachment → working_zlib.cpp

Variants of strange_zlib.cpp showing how to circumvent the problem (and another variant which does fail)

attachment set

rbock@… — Tue, 14 Oct 2008 22:43:07 GMT

attachment → magic.bin

Binary version of short_magic.txt (easier to read into the program, I think, unless it is damaged by upload/download, md5sum: fd78968e81890bce3cee1c3054b700e9)

rbock@… — Wed, 15 Oct 2008 00:16:21 GMT

See also #2411, which shows that the same (or at least very similar) problem occurs for bzip2 compression, too (and much easier to reproduce).

Steven Watanabe — Mon, 14 Jun 2010 19:59:47 GMT

The compressed data is invalid because the stream is not properly ended. Still investigating.

status changed; resolution set

Steven Watanabe — Mon, 14 Jun 2010 21:42:32 GMT

status new → closed
resolution → fixed

(In [62952]) Don't end the stream produced by a reading through a symmetric_filter prematurely. Fixes #2318.