"iostreams::detail::mode_adapter<>" is never "flushable": flushing a filtering_ostream will not flush the "std::ostream" target --- patch included.

[Duncan Exon Smith <duncanphilipnorman@…>]

I have found a problem with Boost.Iostreams.

• I'm using Boost v1.43.0 on Gentoo Linux with gcc-4.3.4.
  • I've reproduced it with gcc-4.4.3.
  • I've reproduced it with Boost v1.44.0 (from the website tarball).
  • Let me know if you need to know anything else about my environment.
• A simple testcases attached at boost_iostreams_filtering_std_ostream.cpp​.
• A patch to fix the problem is attached at boost_iostreams-mode_adaptor-flushable.patch​.
• This is a different problem from #4590.

Details

I'm going to use io as a synonym for the boost::iostreams namespace.

I have come across a problem with io::detail::mode_adapter<Mode, T>, where T is a std::ostream or a std::streambuf. I came across the problem in io::filtering_ostream, but perhaps this class is used elsewhere also.

• io::detail::mode_adapter<>::category is not convertible to any of io::flushable_tag, io::ostream_tag, or io::streambuf_tag.
• io::flush() will use io::detail::flush_device_impl<io::any_tag>::flush() for mode_adapter<, T> even when T::catagory is convertible flushable_tag, ostream_tag or streambuf_tag.
• As a result, io::filtering_ostream will not flush correctly when the device at the end of the chain is a non-boost std::ostream or a std::streambuf.
  • I expect, also, that any filters in the chain that inherit from flushable_tag also do not get flushed correctly.
• In particular the following methods from the STL std::ostream interface will not flush the stream to the device:

  std::ostream stream(&someBuffer);
  
  io::filtering_ostream out;
  out.push(stream);
  
  // These STL methods of flushing a stream will NOT flush "stream".
  out << std::endl;
  out.flush();
  

• My solution is to have mode_adapter<>::category inherit from flushable_tag when appropriate, and to implement ::flush() methods:

[Duncan Exon Smith <duncanphilipnorman@…>]

Patch to fix boost::iostreams::detail::mode_adapter<> to be flushable when appropriate.

[Duncan Exon Smith <duncanphilipnorman@…>]

Testcase to show that a filtering_ostream with a non-boost std::ostream device will not flush.

[Steven Watanabe]

I'm not convinced that stream should be flushed in this case.

[Duncan Exon Smith <duncanphilipnorman@…>]

Replying to steven_watanabe:

I'm not convinced that stream should be flushed in this case.

For clarity, do you mean that you're not convinced the stream should be flushed in the following case?

io::filtering_ostream out;
// Insert 0 or more calls to "out.push(SomeFlushableFilter())" here.
out.push(std::cout);
out << "a b c";
out.flush();

If so, can you give the reasons why not to flush? If not, then can you expand on which case you were talking about?

Here is why I think it should be flushed in the above case:

1. The programmer has asked for the stream to be flushed.
   • Furthermore, std::ostream (thus, std::cout) has the capability of being flushed (it's "flushable"). That it doesn't explicitly have the flushable_tag is simply consequence of it being part of the STL.
     • See boost/iostreams/flush.hpp: boost::flush() will correctly flush std::ostream.
   • In my opinion, the simplest interpretation of the API is that std::cout would be flushed in this case.
     • Why would the library flush std::ostream, but not a wrapper of std::ostream that is an implementation detail? (I previously assumed that this was an overlooked corner case, not an intentional behaviour. Was I wrong? What's the upside?)
     • Is there some other wrapper that the programmer could request use of? I'm thinking something like the following, that perhaps I missed in the documentation:

       out.push(boost::noignoreflush(std::cout));
       

       I maintain that it would be simpler if flushable objects were flushed by default. Something like the following could be used in the rare case that flushing requests should be ignored:

       out.push(boost::noflush(std::cout));
       

2. There isn't a technical challenge (unless my patch missed a corner case; I am not an expert).
3. When a stream is used as a communication channel with another process and timing matters (for example, if there is a handshake, deadlock is bad), not flushing in this case precludes the use of filtering_stream<> with a std::ostream sink.
   • In my opinion, this is an unnecessary restriction on the use of filtering_stream<>.
   • Another example: with the current behaviour, it is dangerous to use filtering_stream<> to add timestamps at the beginning of every line of program output. Even if the programmer reliably flushes the stream, the output cannot be viewed in real time.

[boost@…]

If this patch is not accepted, what is the canonical way of attaching cerr to a filtered stream so that it will flush properly. Is there a work around?

[Duncan Exon Smith <duncanphilipnorman@…>]

I couldn't find a workaround that did not use a patched header.

For my projects, I include a patched copy of mode_adaptor.hpp in an overlay (i.e., $MY_PROJECT/boost/iostreams/detail/adapter/mode_adapter.hpp). Since headers in $MY_PROJECT get precedence over system headers (i.e., I pass -I. to GCC), the patched version gets used automatically.

The bad news is that you need to maintain the patched version of mode_adaptor.hpp yourself when you upgrade Boost.

[Ignacy Gawędzki <boost-bugs@…>]

I happen to have exactly this problem. I applied the patch and it works for me. Could someone review it and reply, please?

Thanks.