Boost C++ Libraries: Ticket #2557: iostreams filtering_stream w/ gzip infinite loop when writing to a full drive

attachment set

Mon, 01 Dec 2008 19:27:47 GMT

attachment → boost-bug.cpp

status changed

Jonathan Turkanis — Mon, 01 Dec 2008 19:58:56 GMT

status new → assigned

michal-slizak@… — Fri, 29 Oct 2010 11:47:16 GMT

Has this been fixed? If so, which version of the library contains the fix?

anonymous — Sat, 30 Oct 2010 12:41:12 GMT

No, it hasn't been fixed.

michal-slizak@… — Wed, 03 Nov 2010 15:23:36 GMT

The gzip file consists of a sequence of compressed chunks.

You can manually compress data chunks and append them to a file:

class StreamSink
{
public:
	typedef char char_type;
	typedef boost::iostreams::sink_tag category;
	StreamSink();
	StreamSink(std::ostream * stream);
	std::streamsize write(const char * ptr, std::streamsize n);
	void close();
	bool isOpen() const;
private:
	bool opened;
	std::ostream * stream;
};
StreamSink::StreamSink():
	opened(false),
	stream(NULL)
{
}
StreamSink::StreamSink(std::ostream * stream):
	opened(true),
	stream(stream)
{
	assert(stream != NULL, "StreamSink constructed from NULL");
}
std::streamsize StreamSink::write(const char * ptr, std::streamsize n)
{
	assert(this->stream != NULL, "Writing to NULL stream");
	this->stream->write(ptr, n);
	return n;
}
void StreamSink::close()
{
	this->opened = false;
	this->stream = NULL;
}
bool StreamSink::isOpen() const
{
	return this->opened;
}
void writeGzippedSection(string const & filename, ios_base::openmode mode, string const & data)
{
	try
	{
		std::ofstream out(filename.c_str(), mode | ios::binary);
		out.exceptions(std::ios_base::badbit | std::ios_base::failbit);
		StreamSink sink(&out);
		boost::iostreams::gzip_compressor compressor;
		compressor.write(sink, data.c_str(), data.length());
		compressor.close(sink, BOOST_IOS::out);
	}
	catch (boost::iostreams::gzip_error const & e)
	{
		std::cerr << "gzip error: " << e.error() << ", zlib error: " << e.zlib_error_code() << std::endl;
		throw;
	}
	catch (ofstream::failure const & e)
	{
		std::cerr << "ofstream::failure: " << e.what() << std::endl;
		throw;
	}
}

The code above produces decompressable files (tested with zcat, zless and gunzip). Also, when out-of-space occurs, chunks that were written successfully can be recovered.

The code may not be pretty, but it works.

leyaya_1999@… — Wed, 15 Jan 2014 15:45:58 GMT

The same infinite loop happens if user does not have write access on the file that is being written to. In that case the variable amt receives value -1 in

/include/boost/iostreams/detail/adapter/non_blocking_adapter.hpp line 42

this bug is still here in boost 1.55

version changed

leyaya_1999@… — Wed, 15 Jan 2014 15:46:47 GMT

version Boost 1.37.0 → Boost 1.55.0

anonymous — Thu, 24 Apr 2014 23:20:18 GMT

And the same issue happens on Windows if the "filename" is longer then MAX_PATH and not prefixed with "\\?\".

anonymous — Tue, 18 Apr 2017 13:57:47 GMT

There are also hangs if a file_sink could not be opened (though that is at least possible to check manually by calling is_open() before). Also, the "hangs" are not just hangs but infinite loops with 100% CPU usage. It feels like the whole iostream was written without any consideration for error handling at all.

anonymous — Mon, 03 Jul 2017 13:11:20 GMT

Submitted a pull request with fix and test-case: https://github.com/boostorg/iostreams/pull/36

It doesn't check or ensure that the error is properly propagated (I am not sure there even is a proper way to do that), but at least no more infinite loop.

anonymous — Wed, 12 Jul 2017 12:45:47 GMT

Change has been merged, so hopefully the next release will finally have the fix.