bzip2_decompressor does not work properly with a filtering_streambuf

[Chris Steenwyk <chris.steenwyk@…>]

The revision 63057 of boost/iostreams/filter/bzip2.hpp seems to have broken the ability to use the bzip2 decompressor in a filtering streambuf. When I pass a stream into the buffer it throws data_error_magic. Stepping through the code I can see it correctly decode the zipped data, but then it continues to attempt to decode data even after eof is reached.

Looking at the changes to bzip2_decompressor_impl I see the following issues: 1.) The function returns true even if _eof is set to true. This causes the "done" flag in the "Read" function in symmetric.hpp to be set to false, which causes the bzip2_decompressor_impl function to execute again, after EOL. 2.) On the second execution when _eof is set, the "close" function resets _eof to false, which can cause the function to throw the exception.

In the specific instance I am using, my bzip compressed section is embedded in the middle of the stream, so src_begin won't equal src_end.

[anonymous]

Can you provide a code sample that demonstrates the problem? Even better, look at the existing bzip2 tests in boost\libs\iostreams\test\bzip2_test.cpp and see if you can add a test case which demonstrates it.

[chris.steenwyk@…]

Patch file for libs/iostreams/test/bzip2_test.cpp

[chris.steenwyk@…]

Sorry it took so long.

The code below works on boost 1.42 and generates an exception on 1.45. I have also attached a patch to add this test into the test file bzip2_test.cpp.

#include <string>
#include <boost/iostreams/filter/bzip2.hpp>
#include <boost/iostreams/filter/test.hpp>
#include <boost/iostreams/filtering_stream.hpp>

std::vector<char> temp;
std::string source( "This is test data" );
std::vector<char> sourceVector( source.begin(), source.end() );
std::vector<char> dest;
	
// Write compressed source to temp
boost::iostreams::filtering_streambuf< boost::iostreams::output > out;
out.push( boost::iostreams::bzip2_compressor() );
out.push( boost::iostreams::back_inserter( temp ) );
boost::iostreams::copy( boost::make_iterator_range( source ), out );
	
// Add uncompressed source around the compressed source
temp.insert( temp.begin(), source.begin(), source.end() );
temp.insert( temp.end(), source.begin(), source.end() );
			
// Read compressed data from temp into dest
boost::iostreams::array_source inStr( &temp[ source.length() ], temp.size() );
boost::iostreams::filtering_streambuf< boost::iostreams::input > in;
in.push( boost::iostreams::bzip2_decompressor() );
in.push( inStr );
boost::iostreams::copy( in, boost::iostreams::back_inserter( dest ) );

[vograno@…]

I ran into the same problem. Boost version 1.49. Is there a workaround? Any alternative way to decompress bzip2?

[Fred Baba <fred.baba@…>]

Same issue in Boost 1.53.