Boost C++ Libraries: Ticket #8150: gzip_compressor produces corrupt data when used with filtering_ostream https://svn.boost.org/trac10/ticket/8150 <p> In some cases, the USE_CORRUPTING_OSTREAM path in the following program will produce corrupt gzip output (fails CRC and the de-gzipped file differs slightly from the original input). This does not occur for all data, so I will attach an input data file that triggers the error. </p> <p> This bug only affects gzip_stream when used with filtering_ostream. It does not affect filtering_istream usage. </p> <p> Bug reproduced on Mac 10.8.2, 64-bit, Apple LLVM version 4.2 (clang-425.0.24) (based on LLVM 3.2svn), zlib 1.2.5 (dylib as shipped with Mac OS X), boost 1.53.0. </p> <p> Bug does not appear to be affected by compiler optimizations (-O0 and -Os tested). </p> <pre class="wiki">#include &lt;iostream&gt; #include &lt;fstream&gt; #include &lt;vector&gt; #include &lt;boost/iostreams/filter/gzip.hpp&gt; #include &lt;boost/iostreams/filtering_stream.hpp&gt; #include &lt;boost/iostreams/device/back_inserter.hpp&gt; #include &lt;boost/iostreams/copy.hpp&gt; #include &lt;boost/iostreams/device/array.hpp&gt; #include &lt;boost/iostreams/stream.hpp&gt; #include &lt;boost/iostreams/device/file_descriptor.hpp&gt; int main(int argc, char *argv[]) { std::vector&lt;char&gt; input; std::vector&lt;char&gt; output; boost::iostreams::file_descriptor_source source("inputdata"); boost::iostreams::copy(source, boost::iostreams::back_inserter(input)); #define USE_CORRUPTING_OSTREAM 1 #if USE_CORRUPTING_OSTREAM boost::iostreams::filtering_ostream gzip_stream; gzip_stream.push(boost::iostreams::gzip_compressor()); gzip_stream.push(boost::iostreams::back_inserter(output)); gzip_stream.write(&amp;input[0], input.size()); boost::iostreams::close(gzip_stream); #else boost::iostreams::filtering_istream gzip_stream; boost::iostreams::stream&lt;boost::iostreams::array_source&gt; input_array(&amp;input[0], input.size()); gzip_stream.push(boost::iostreams::gzip_compressor()); gzip_stream.push(input_array); boost::iostreams::copy(gzip_stream, boost::iostreams::back_inserter(output)); #endif boost::iostreams::file_descriptor_sink destination("inputdata.gz"); destination.write(&amp;output[0], output.size()); } </pre> en-us Boost C++ Libraries /htdocs/site/boost.png https://svn.boost.org/trac10/ticket/8150 Trac 1.4.3 matt@… Sat, 23 Feb 2013 00:01:17 GMT attachment set https://svn.boost.org/trac10/ticket/8150 https://svn.boost.org/trac10/ticket/8150 <ul> <li><strong>attachment</strong> → <span class="trac-field-new">inputdata</span> </li> </ul> <p> inputdata to be used with the program in the ticket to trigger the bug </p> Ticket matt@… Mon, 25 Feb 2013 11:49:51 GMT <link>https://svn.boost.org/trac10/ticket/8150#comment:1 </link> <guid isPermaLink="false">https://svn.boost.org/trac10/ticket/8150#comment:1</guid> <description> <p> To be specific, when this inputdata file is run through the two different code paths, the only difference is that the USE_CORRUPTING_OSTREAM version is one byte shorter -- it is missing the 0xFF at byte 0x4000. Otherwise, the gzipped output is identical. </p> </description> <category>Ticket</category> </item> <item> <author>matt@…</author> <pubDate>Mon, 25 Feb 2013 15:07:55 GMT</pubDate> <title/> <link>https://svn.boost.org/trac10/ticket/8150#comment:2 </link> <guid isPermaLink="false">https://svn.boost.org/trac10/ticket/8150#comment:2</guid> <description> <p> Further investigation has revealed that this is a bug in libcxx, not boost. The fix is here: </p> <p> <a class="ext-link" href="https://llvm.org/viewvc/llvm-project/libcxx/trunk/include/streambuf?annotate=165884"><span class="icon">​</span>https://llvm.org/viewvc/llvm-project/libcxx/trunk/include/streambuf?annotate=165884</a> </p> <p> basically, the 0xff in the data was incorrectly passed by libcxx through to indirect_streambuf::overflow, causing it to be interpreted as an EOF (and rejected). </p> <p> Please close this bug. </p> </description> <category>Ticket</category> </item> </channel> </rss>