A particular string for which compressing/uncompressing using iostreams/zlib fails.

[guy_tannenbaum@…]

Found a string that compressing and than uncompressing using iostreams and zlib results in an exception being thrown by the uncompress operation. (The same code works fine for all other strings tried). Attaching a sample program + file containing the problematic string.

Not sure if this helps - but take note that the resulting compressed buffer has the pretty unusual size of 20480, which might be a multiple of some constant. A number of different strings which compress to this size however didn't exhibit the same behavior.

[guy_tannenbaum@…]

program which generates the behavior

[guy_tannenbaum@…]

problematic string (each line is the ascii code of one char)

[guy_tannenbaum@…]

Found this on Boost 1.35.0 (not 1.36.0)

[rbock@…]

Hi,

I found the same problem in 1.36. It is rather rare: In 6.5Million compressed files in my project, it occurred only 441 times. As mentioned by Guy, the size of the damaged zip is a bit peculiar, all sizes I checked are multiples of 4096.

Also, I think it is more related to filtering_istream and filtering_istreambuf than zlib_compression, as it works fine if you use filtering_ostream or filtering_ostreambuf instead.

I am going to attach an extended example with similar code which does not work, too for the given example string and code which does (using filtering_ostream instead of filtering_istream for compression).

I am also attaching a binary version of the sample data, since reading the ascii version did not work for me (I had to replace the lexical_cast by atoi to make it work).

Here is some system information, if this should be required to reproduce the problem (and circumvention):

Ubuntu 8.04 64bit, gcc 4.2.3, boost-1.36, zlib-1g 1.2.3.3

Please do not hesitate to ask for more information. Personally, I consider this a rather severe problem since it can impair data integrity in backups. Lucky me, my software is not in production yet :-)

Regards,

Roland

[rbock@…]

Variants of strange_zlib.cpp showing how to circumvent the problem (and another variant which does fail)

[rbock@…]

Binary version of short_magic.txt (easier to read into the program, I think, unless it is damaged by upload/download, md5sum: fd78968e81890bce3cee1c3054b700e9)

[rbock@…]

See also #2411, which shows that the same (or at least very similar) problem occurs for bzip2 compression, too (and much easier to reproduce).

[Steven Watanabe]

The compressed data is invalid because the stream is not properly ended. Still investigating.

[Steven Watanabe]

(In [62952]) Don't end the stream produced by a reading through a symmetric_filter prematurely. Fixes #2318.