Boost C++ Libraries: Ticket #3853: bzip2_decompressor reads only one block of a pbzip2-created files

attachment set

Thu, 21 Jan 2010 08:50:51 GMT

attachment → spit.cc

creates test file

attachment set

Thu, 21 Jan 2010 08:52:22 GMT

attachment → test_bzip2_decompressor.cc

decompressor using boost iostream filter

Thu, 21 Jan 2010 08:56:10 GMT

sorry, the line in the original bug description got concatenated. it should read:

c++ spit.cc -o spit

c++ test_bzip2_decompressor.cc -o test_bzip2_decompressor

./spit >test.txt

bzip2 -c test.txt >test.txt.bz2

pbzip2 -c test.txt >test.txt.parallel.bz2

jeff.gilchrist@… — Wed, 27 Jan 2010 17:50:49 GMT

It seems like boost does not support multiple bz2 streams in a file. Both bzip2 and pbzip2 support reading multiple bz2 streams, so when you see the end of a bz2 sequence, you have check if it is the EOF as well, if not, look for another bz2 sequence and concatenate the decompressed results if one is found.

So you can have files like:

|bz1|

|bz1|bz2|bz3|bz4| etc...

And you need to support both if you want to be compatible with bzip2 and pbzip2 since they both support this and it is a valid format.

status changed; resolution set

Steven Watanabe — Thu, 17 Jun 2010 19:14:17 GMT

status new → closed
resolution → fixed

(In [63057]) Allow bzip2_decompressor to process multiple concatenated streams. Fixes #3853.