Boost C++ Libraries: Ticket #4590: Flushing a filtering_ostream stopped working in Boost 1.44

attachment set

t0rt1e@… — Wed, 25 Aug 2010 10:29:49 GMT

attachment → boost_iostreams_filtering_ostream.cpp

Test case

tort1e@… — Wed, 25 Aug 2010 13:08:30 GMT

Maybe I should mention the platform, on which I observed this behavior: Mac OS X 10.6 x86_64 with Boost 1.44 installed through MacPorts.

attachment set

duncanphilipnorman@… — Wed, 08 Sep 2010 19:44:10 GMT

attachment → boost_iostreams_filtering_ostream_does_not_flush_at_all.cpp

Testcase for linux that shows that even cout does not actually get flushed.

cc set

duncanphilipnorman@… — Wed, 08 Sep 2010 19:53:03 GMT

cc duncanphilipnorman@… added

I've attached boost_iostreams_filtering_ostream_does_not_flush_at_all.cpp , which demonstrates (on Linux) that even std::cout does not get flushed by filtering_ostream.

I'm using:

GCC 4.3.4
Gentoo Linux
Boost 1.43.0

I'm not sure if this is the same problem, but my post to the mailing list got no response.

The basic recipe:

filtering_ostream out;
out.push(any_filter_at_all());
out.push(std::cout);
out << "Hello World!" << std::flush;
sleep(10); // Observe an empty console.
std::cout << std::flush;
sleep(10); // Observe printed text on console.

This problem is quite bad for me when doing the following on the command-line (for a more complicated program):
```
 ./program 2>&1 | tee some-file.log
```
Because stdout is not getting flushed, the stdout and stderr get intermingled strangely.
However, disabling buffering on stdout works around the problem:
```
 stdbuf --output=0 ./program 2>&1 | tee some-file.log
```

anonymous — Wed, 08 Sep 2010 19:57:00 GMT

Replying to duncanphilipnorman@…:

I've attached boost_iostreams_filtering_ostream_does_not_flush_at_all.cpp , which demonstrates (on Linux) that even std::cout does not get flushed by filtering_ostream.

By the way, my test program is much less elegant than the original reporter's testcase. One must observe console output (based on timing due to sleep() calls) to see the problem, since I wasn't sure how to test std::cout otherwise.

Fri, 24 Sep 2010 18:04:11 GMT

Dear all,

I recently upgraded from Boost 1.43.0 to 1.44.0 and have experienced the same problem. My personal debug logging library relies upon the proper functioning of the flush (and sync) mechanism in boost::iostreams to produce transactional, orderly message output. Unfortunately, in the latest boost version this flush mechanism is non-functional.

However, I've made an investigation of this problem by putting versions 1.43.0 and 1.44.0 side-by-side. I am relieved to say that I have tracked the problem, and have a simple solution.

I have confirmed that this problem was introduced by the bugfix for #2356, which changed the following code in boost/iostreams/detail/adapter/concept_adapter.hpp:

Boost v1.43.0

  bool flush( BOOST_IOSTREAMS_BASIC_STREAMBUF(char_type,
              BOOST_IOSTREAMS_CHAR_TRAITS(char_type))* sb )
  {
      bool result = any_impl::flush(t_, sb);
      if (sb && sb->BOOST_IOSTREAMS_PUBSYNC() == -1)
          result = false;
      return result;
  }

Boost v1.44.0

  template<typename Device>
  bool flush( Device* dev )
  {
      return any_impl::flush(t_, dev);
  }

(See changeset: https://svn.boost.org/trac/boost/changeset/63034.)

For some reason, the 'PUBSYNC' operation was removed. I suspect that it was merely mistaken as being unnecessary, however it is the mechanism by which the filtering_ostream's chain is flushed in its entirety.

To give some context: indirect_streambuf (the stream buffer used by filtering_ostream), being a std::streambuf, has a required 'sync' method:

boost::iostreams::indirect_detail::indirect_streambuf::sync

  template<typename T, typename Tr, typename Alloc, typename Mode>
  int indirect_streambuf<T, Tr, Alloc, Mode>::sync()
  {
      try { // sync() is no-throw.
          sync_impl();
          obj().flush(next_);
          return 0;
      } catch (...) { return -1; }
  }

Where 'obj()' returns a 'concept_adapter<T>&'. This 'sync' method's implementation has not changed between versions 1.43.0 and 1.44.0. This sync method is called by ostream::flush, and relies upon the expectation that any 'flush' will indeed itself perform a 'sync' operation (the aforementioned 'PUBSYNC').

See: http://www.cplusplus.com/reference/iostream/ostream/flush/
And: http://www.cplusplus.com/reference/iostream/streambuf/pubsync/

However, as 'PUBSYNC' has been removed from concept_adaptor<T>::sync, only the first device or filter in the filtering_ostream's chain will ever be flushed. This means that for any filtering_ostream which a chain of length 2 or more, the stream's output will only reach its destination when a buffer overflows, or the stream is released.

So, I hope that by now the solution is self-evident:

Proposal for Boost 1.45.0

  template<typename Device>
  bool flush( Device* dev )
  {
      bool result = any_impl::flush(t_, dev);
      if (dev && dev->BOOST_IOSTREAMS_PUBSYNC() == -1)
          result = false;
      return result;
  }

I've tested this solution with the test code provided both in this bug report (attachment: boost_iostreams_filtering_ostream_does_not_flush_at_all.cpp), and #2356 (attachment: test.cpp).

I'm not myself a boost contributor (this is my first contribution), so if you believe it to be correct, could someone please submit this change to SVN? Correction diff file to follow.

Many thanks.

attachment set

Fri, 24 Sep 2010 18:06:06 GMT

attachment → boost_4590_correction.diff

Correction to concept_adapter.hpp for boost 1.44.0.

Tue, 12 Oct 2010 21:56:03 GMT

Replying to duncanphilipnorman@…:

I've attached boost_iostreams_filtering_ostream_does_not_flush_at_all.cpp , which demonstrates (on Linux) that even std::cout does not get flushed by filtering_ostream.

I believe my issue is distinct from the one described by the original poster -- particularly because my issue is present already in Boost 1.43.0. I have created a new ticket for my issue (see #4728), along with a patch to fix it.

status changed; resolution set

Steven Watanabe — Fri, 11 Feb 2011 16:18:38 GMT

status new → closed
resolution → fixed

(In [68779]) Fix flush regression. Fixes #4590.