Opened 12 years ago

Closed 12 years ago

#4590 closed Bugs (fixed)

Flushing a filtering_ostream stopped working in Boost 1.44

Reported by: t0rt1e@… Owned by: Jonathan Turkanis
Milestone: To Be Determined Component: iostreams
Version: Boost 1.44.0 Severity: Problem
Keywords: Cc: duncanphilipnorman@…

Description

Since the update to Boost 1.44, flushing an filtering_ostream containing an output_filter and a sink different from std::cout (tested boost::iostreams::back_inserter(std::string) and boost::test_tools::output_test_stream) using the flush() member function stopped working. The output string stays empty. This used to work until Boost 1.43.

I tried to replace the call of flush() with the sync() member function documented in <http://www.boost.org/doc/libs/1_44_0/libs/iostreams/doc/classes/filtering_stream.html>, which leads to compiler error that filtering_ostream does not have this member function. Only using the member function strict_sync() flushes correctly the filtering_ostream in Boost 1.44.

Attached is a test case based on the back_inserter example in <http://www.boost.org/doc/libs/1_44_0/libs/iostreams/doc/tutorial/container_sink.html> augmented by a transparent filter simplified from the code example in <http://www.boost.org/doc/libs/1_44_0/libs/iostreams/doc/concepts/output_filter.html>.

Attachments (3)

boost_iostreams_filtering_ostream.cpp​ (2.5 KB ) - added by t0rt1e@… 12 years ago.
Test case
boost_iostreams_filtering_ostream_does_not_flush_at_all.cpp​ (2.3 KB ) - added by duncanphilipnorman@… 12 years ago.
Testcase for linux that shows that even cout does not actually get flushed.
boost_4590_correction.diff​ (539 bytes ) - added by Mike Tryhorn <miketryhorn@…> 12 years ago.
Correction to concept_adapter.hpp for boost 1.44.0.

Download all attachments as: .zip

Change History (9)

by t0rt1e@…, 12 years ago

Attachment: boost_iostreams_filtering_ostream.cpp​ added

Test case

comment:1 by tort1e@…, 12 years ago

Maybe I should mention the platform, on which I observed this behavior: Mac OS X 10.6 x86_64 with Boost 1.44 installed through MacPorts.

by duncanphilipnorman@…, 12 years ago

Attachment: boost_iostreams_filtering_ostream_does_not_flush_at_all.cpp​ added

Testcase for linux that shows that even cout does not actually get flushed.

comment:2 by duncanphilipnorman@…, 12 years ago

Cc: duncanphilipnorman@… added

I've attached boost_iostreams_filtering_ostream_does_not_flush_at_all.cpp​, which demonstrates (on Linux) that even std::cout does not get flushed by filtering_ostream.

I'm using:

  • GCC 4.3.4
  • Gentoo Linux
  • Boost 1.43.0

I'm not sure if this is the same problem, but ​my post to the mailing list got no response.

  1. The basic recipe: 
    filtering_ostream out;
out.push(any_filter_at_all());
out.push(std::cout);
out << "Hello World!" << std::flush;
sleep(10); // Observe an empty console.
std::cout << std::flush;
sleep(10); // Observe printed text on console.
  2. This problem is quite bad for me when doing the following on the command-line (for a more complicated program): 
     ./program 2>&1 | tee some-file.log
    Because stdout is not getting flushed, the stdout and stderr get intermingled strangely.
  3. However, disabling buffering on stdout works around the problem: 
     stdbuf --output=0 ./program 2>&1 | tee some-file.log

in reply to:  2 comment:3 by anonymous, 12 years ago

Replying to duncanphilipnorman@…:

I've attached boost_iostreams_filtering_ostream_does_not_flush_at_all.cpp​, which demonstrates (on Linux) that even std::cout does not get flushed by filtering_ostream.

By the way, my test program is much less elegant than the original reporter's testcase. One must observe console output (based on timing due to sleep() calls) to see the problem, since I wasn't sure how to test std::cout otherwise.

comment:4 by Mike Tryhorn <miketryhorn@…>, 12 years ago

Dear all,

I recently upgraded from Boost 1.43.0 to 1.44.0 and have experienced the same problem. My personal debug logging library relies upon the proper functioning of the flush (and sync) mechanism in boost::iostreams to produce transactional, orderly message output. Unfortunately, in the latest boost version this flush mechanism is non-functional.

However, I've made an investigation of this problem by putting versions 1.43.0 and 1.44.0 side-by-side. I am relieved to say that I have tracked the problem, and have a simple solution.

I have confirmed that this problem was introduced by the bugfix for #2356, which changed the following code in boost/iostreams/detail/adapter/concept_adapter.hpp:

Boost v1.43.0 

  bool flush( BOOST_IOSTREAMS_BASIC_STREAMBUF(char_type,
              BOOST_IOSTREAMS_CHAR_TRAITS(char_type))* sb )
  {
      bool result = any_impl::flush(t_, sb);
      if (sb && sb->BOOST_IOSTREAMS_PUBSYNC() == -1)
          result = false;
      return result;
  }

Boost v1.44.0 

  template<typename Device>
  bool flush( Device* dev )
  {
      return any_impl::flush(t_, dev);
  }

(See changeset: ​https://svn.boost.org/trac/boost/changeset/63034.)

For some reason, the 'PUBSYNC' operation was removed. I suspect that it was merely mistaken as being unnecessary, however it is the mechanism by which the filtering_ostream's chain is flushed in its entirety.

To give some context: indirect_streambuf (the stream buffer used by filtering_ostream), being a std::streambuf, has a required 'sync' method:

boost::iostreams::indirect_detail::indirect_streambuf::sync 

  template<typename T, typename Tr, typename Alloc, typename Mode>
  int indirect_streambuf<T, Tr, Alloc, Mode>::sync()
  {
      try { // sync() is no-throw.
          sync_impl();
          obj().flush(next_);
          return 0;
      } catch (...) { return -1; }
  }

Where 'obj()' returns a 'concept_adapter<T>&'. This 'sync' method's implementation has not changed between versions 1.43.0 and 1.44.0. This sync method is called by ostream::flush, and relies upon the expectation that any 'flush' will indeed itself perform a 'sync' operation (the aforementioned 'PUBSYNC').

See: ​http://www.cplusplus.com/reference/iostream/ostream/flush/
And: ​http://www.cplusplus.com/reference/iostream/streambuf/pubsync/

However, as 'PUBSYNC' has been removed from concept_adaptor<T>::sync, only the first device or filter in the filtering_ostream's chain will ever be flushed. This means that for any filtering_ostream which a chain of length 2 or more, the stream's output will only reach its destination when a buffer overflows, or the stream is released.

So, I hope that by now the solution is self-evident:

Proposal for Boost 1.45.0 

  template<typename Device>
  bool flush( Device* dev )
  {
      bool result = any_impl::flush(t_, dev);
      if (dev && dev->BOOST_IOSTREAMS_PUBSYNC() == -1)
          result = false;
      return result;
  }

I've tested this solution with the test code provided both in this bug report (attachment: boost_iostreams_filtering_ostream_does_not_flush_at_all.cpp), and #2356 (attachment: test.cpp).

I'm not myself a boost contributor (this is my first contribution), so if you believe it to be correct, could someone please submit this change to SVN? Correction diff file to follow.

Many thanks.

by Mike Tryhorn <miketryhorn@…>, 12 years ago

Attachment: boost_4590_correction.diff​ added

Correction to concept_adapter.hpp for boost 1.44.0.

in reply to:  2 comment:5 by Duncan Exon Smith <duncanphilipnorman@…>, 12 years ago

Replying to duncanphilipnorman@…:

I've attached boost_iostreams_filtering_ostream_does_not_flush_at_all.cpp​, which demonstrates (on Linux) that even std::cout does not get flushed by filtering_ostream.

I believe my issue is distinct from the one described by the original poster -- particularly because my issue is present already in Boost 1.43.0. I have created a new ticket for my issue (see #4728), along with a patch to fix it.

comment:6 by Steven Watanabe, 12 years ago

Resolution: → fixed
Status: new → closed

(In [68779]) Fix flush regression. Fixes #4590.

Note: See TracTickets for help on using tickets.