Boost C++ Libraries: Ticket #13515: async_pipe::async_read_some returns zero read size if -O<something> is used https://svn.boost.org/trac10/ticket/13515 <p> Hi there, </p> <p> We are developing coroutine based application which sits mostly on some sort of I/O. Decided to go with coroutine based approach with boost 1.66 and asio/beast/process libs. While doing some internal POC we ran into issue with retrieving size of data obtained through pipe. </p> <div class="wikipage" style="font-size: 80%"><div class="wiki-code"><div class="code"><pre><span class="cp">#include</span> <span class="cpf">&lt;boost/asio.hpp&gt;</span><span class="cp"></span> <span class="cp">#include</span> <span class="cpf">&lt;boost/asio/spawn.hpp&gt;</span><span class="cp"></span> <span class="cp">#include</span> <span class="cpf">&lt;boost/process.hpp&gt;</span><span class="cp"></span> <span class="cp">#include</span> <span class="cpf">&lt;chrono&gt;</span><span class="cp"></span> <span class="cp">#include</span> <span class="cpf">&lt;iostream&gt;</span><span class="cp"></span> <span class="kt">void</span> <span class="nf">test_process</span><span class="p">()</span> <span class="p">{</span> <span class="k">static</span> <span class="k">const</span> <span class="kt">char</span><span class="o">*</span> <span class="n">path</span> <span class="o">=</span> <span class="s">&quot;./dataspit.py&quot;</span><span class="p">;</span> <span class="k">static</span> <span class="k">const</span> <span class="kt">char</span><span class="o">*</span> <span class="n">interpreter</span> <span class="o">=</span> <span class="s">&quot;python3&quot;</span><span class="p">;</span> <span class="n">std</span><span class="o">::</span><span class="n">cout</span> <span class="o">&lt;&lt;</span> <span class="s">&quot;Hello from streamer2</span><span class="se">\n</span><span class="s">&quot;</span><span class="p">;</span> <span class="k">namespace</span> <span class="n">asio</span> <span class="o">=</span> <span class="n">boost</span><span class="o">::</span><span class="n">asio</span><span class="p">;</span> <span class="k">namespace</span> <span class="n">process</span> <span class="o">=</span> <span class="n">boost</span><span class="o">::</span><span class="n">process</span><span class="p">;</span> <span class="k">namespace</span> <span class="n">chrono</span> <span class="o">=</span> <span class="n">std</span><span class="o">::</span><span class="n">chrono</span><span class="p">;</span> <span class="n">asio</span><span class="o">::</span><span class="n">io_context</span> <span class="n">ioc</span><span class="p">;</span> <span class="n">asio</span><span class="o">::</span><span class="n">spawn</span><span class="p">(</span><span class="n">ioc</span><span class="p">,</span> <span class="p">[</span><span class="o">&amp;</span><span class="n">ioc</span><span class="p">](</span><span class="n">asio</span><span class="o">::</span><span class="n">yield_context</span> <span class="n">yield</span><span class="p">)</span> <span class="p">{</span> <span class="n">process</span><span class="o">::</span><span class="n">async_pipe</span> <span class="n">pipe</span><span class="p">{</span><span class="n">ioc</span><span class="p">};</span> <span class="k">auto</span> <span class="n">child</span> <span class="o">=</span> <span class="n">process</span><span class="o">::</span><span class="n">child</span><span class="p">{</span><span class="n">process</span><span class="o">::</span><span class="n">search_path</span><span class="p">(</span><span class="n">interpreter</span><span class="p">),</span> <span class="n">path</span><span class="p">,</span> <span class="n">process</span><span class="o">::</span><span class="n">std_out</span> <span class="o">&gt;</span> <span class="n">pipe</span><span class="p">};</span> <span class="n">std</span><span class="o">::</span><span class="n">array</span><span class="o">&lt;</span><span class="kt">char</span><span class="p">,</span> <span class="mi">4096</span><span class="o">&gt;</span> <span class="n">buffer</span><span class="p">{</span><span class="mi">0</span><span class="p">};</span> <span class="n">std</span><span class="o">::</span><span class="n">cout</span> <span class="o">&lt;&lt;</span> <span class="s">&quot;Buffer state: &quot;</span> <span class="o">&lt;&lt;</span> <span class="p">(</span><span class="kt">int</span><span class="p">)</span><span class="n">buffer</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="o">&lt;&lt;</span> <span class="p">(</span><span class="kt">int</span><span class="p">)</span><span class="n">buffer</span><span class="p">[</span><span class="mi">1</span><span class="p">]</span> <span class="o">&lt;&lt;</span> <span class="n">std</span><span class="o">::</span><span class="n">endl</span><span class="p">;</span> <span class="n">boost</span><span class="o">::</span><span class="n">system</span><span class="o">::</span><span class="n">error_code</span> <span class="n">read_ec</span><span class="p">;</span> <span class="k">do</span> <span class="p">{</span> <span class="k">auto</span> <span class="n">timePoint</span> <span class="o">=</span> <span class="n">chrono</span><span class="o">::</span><span class="n">system_clock</span><span class="o">::</span><span class="n">now</span><span class="p">();</span> <span class="n">std</span><span class="o">::</span><span class="n">cout</span> <span class="o">&lt;&lt;</span> <span class="n">timePoint</span><span class="p">.</span><span class="n">time_since_epoch</span><span class="p">().</span><span class="n">count</span><span class="p">()</span> <span class="o">&lt;&lt;</span> <span class="s">&quot;: Sleeping</span><span class="se">\n</span><span class="s">&quot;</span><span class="p">;</span> <span class="n">std</span><span class="o">::</span><span class="kt">size_t</span> <span class="n">size</span> <span class="o">=</span> <span class="n">pipe</span><span class="p">.</span><span class="n">async_read_some</span><span class="p">(</span><span class="n">asio</span><span class="o">::</span><span class="n">buffer</span><span class="p">(</span><span class="n">buffer</span><span class="p">),</span> <span class="n">yield</span><span class="p">[</span><span class="n">read_ec</span><span class="p">]);</span> <span class="k">if</span> <span class="p">(</span><span class="n">read_ec</span><span class="p">)</span> <span class="p">{</span> <span class="n">std</span><span class="o">::</span><span class="n">cerr</span> <span class="o">&lt;&lt;</span> <span class="n">read_ec</span><span class="p">.</span><span class="n">message</span><span class="p">()</span> <span class="o">&lt;&lt;</span> <span class="n">std</span><span class="o">::</span><span class="n">endl</span><span class="p">;</span> <span class="k">break</span><span class="p">;</span> <span class="p">}</span> <span class="n">timePoint</span> <span class="o">=</span> <span class="n">chrono</span><span class="o">::</span><span class="n">system_clock</span><span class="o">::</span><span class="n">now</span><span class="p">();</span> <span class="n">std</span><span class="o">::</span><span class="n">cout</span> <span class="o">&lt;&lt;</span> <span class="n">timePoint</span><span class="p">.</span><span class="n">time_since_epoch</span><span class="p">().</span><span class="n">count</span><span class="p">()</span> <span class="o">&lt;&lt;</span> <span class="s">&quot;: Read &quot;</span> <span class="o">&lt;&lt;</span> <span class="n">size</span> <span class="o">&lt;&lt;</span> <span class="s">&quot; bytes</span><span class="se">\n</span><span class="s">&quot;</span><span class="p">;</span> <span class="n">std</span><span class="o">::</span><span class="n">cout</span> <span class="o">&lt;&lt;</span> <span class="s">&quot;Buffer state: &quot;</span> <span class="o">&lt;&lt;</span> <span class="p">(</span><span class="kt">int</span><span class="p">)</span><span class="n">buffer</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="o">&lt;&lt;</span> <span class="p">(</span><span class="kt">int</span><span class="p">)</span><span class="n">buffer</span><span class="p">[</span><span class="mi">1</span><span class="p">]</span> <span class="o">&lt;&lt;</span> <span class="n">std</span><span class="o">::</span><span class="n">endl</span><span class="p">;</span> <span class="p">}</span> <span class="k">while</span> <span class="p">(</span><span class="n">read_ec</span> <span class="o">!=</span> <span class="n">boost</span><span class="o">::</span><span class="n">asio</span><span class="o">::</span><span class="n">error</span><span class="o">::</span><span class="n">eof</span><span class="p">);</span> <span class="n">child</span><span class="p">.</span><span class="n">wait</span><span class="p">();</span> <span class="p">}</span> <span class="p">);</span> <span class="n">ioc</span><span class="p">.</span><span class="n">run</span><span class="p">();</span> <span class="n">std</span><span class="o">::</span><span class="n">cout</span> <span class="o">&lt;&lt;</span> <span class="s">&quot;Farewell from streamer2</span><span class="se">\n</span><span class="s">&quot;</span><span class="p">;</span> <span class="p">}</span> <span class="kt">int</span> <span class="nf">main</span><span class="p">(</span><span class="kt">int</span> <span class="n">argc</span><span class="p">,</span> <span class="kt">char</span><span class="o">**</span> <span class="n">argv</span><span class="p">)</span> <span class="p">{</span> <span class="n">test_process</span><span class="p">();</span> <span class="k">return</span> <span class="mi">0</span><span class="p">;</span> <span class="p">}</span> </pre></div></div></div><p> This code works fine when no compiler optimizations are used. As soon as its compiled with -Os or any numbered -O type. </p> <div class="wikipage" style="font-size: 80%"><div class="wiki-code"><div class="code"><pre><span class="n">pipe</span><span class="p">.</span><span class="n">async_read_some</span><span class="p">(</span><span class="n">asio</span><span class="o">::</span><span class="n">buffer</span><span class="p">(</span><span class="n">buffer</span><span class="p">),</span> <span class="n">yield</span><span class="p">[</span><span class="n">read_ec</span><span class="p">]);</span> </pre></div></div></div><p> starts returning 0. Buffer contents are modified though. Callback based approach yields same result. </p> <div class="wikipage" style="font-size: 80%"><pre class="wiki">... readv(6, [{"57757488835088650590974423544437"..., 4096}], 1) = 1030 write(1, "1523018074402113597: Read 0 byte"..., 341523018074402113597: Read 0 bytes) = 34 write(1, "Buffer state: 5355\n", 19Buffer state: 5355) = 19 write(1, "1523018074402457408: Sleeping\n", 301523018074402457408: Sleeping) = 30 readv(6, 0x83c2168, 1) = -1 EAGAIN (Resource temporarily unavailable) epoll_wait(4, [{EPOLLIN, {u32=138163408, u64=138163408}}], 128, -1) = 1 readv(6, [{"15827357453760420053575823774839"..., 4096}], 1) = 546 write(1, "1523018076807802498: Read 0 byte"..., 341523018076807802498: Read 0 bytes) = 34 write(1, "Buffer state: 4953\n", 19Buffer state: 4953) = 19 write(1, "1523018076808233410: Sleeping\n", 301523018076808233410: Sleeping) = 30 readv(6, 0x83c2168, 1) = -1 EAGAIN (Resource temporarily unavailable) epoll_wait(4, [{EPOLLIN, {u32=138163408, u64=138163408}}], 128, -1) = 1 readv(6, [{"66365484273136085944223051047033"..., 4096}], 1) = 814 write(1, "1523018079517434883: Read 0 byte"..., 341523018079517434883: Read 0 bytes) = 34 write(1, "Buffer state: 5454\n", 19Buffer state: 5454) = 19 write(1, "1523018079517782099: Sleeping\n", 301523018079517782099: Sleeping) = 30 readv(6, 0x83c2168, 1) = -1 EAGAIN (Resource temporarily unavailable) ... </pre></div><p> Epoll waits as expected, readv seems to return length as well. Looks like some kind of reordering issue. </p> <p> Tested this behavior on ubuntu 14.04/16.04 in 32/64bit modes on using stock gcc, all ran in docker containers within ubuntu 16.04 host. </p> <p> Issue applies to boost 1.66 for sure, tested also 1.65 and 1.67 beta on ubuntu 14.04 32 bit. Issue is there as well. </p> <p> Complete sample code in attachments. </p> <p> I'd be grateful for some notification if you find root cause and possible patch. I'd like to apply such to our boost sources. </p> <p> Best Regards, Tomasz Jonak </p> en-us Boost C++ Libraries /htdocs/site/boost.png https://svn.boost.org/trac10/ticket/13515 Trac 1.4.3 Tomasz Jonak <tjonak@…> Fri, 06 Apr 2018 14:20:28 GMT attachment set https://svn.boost.org/trac10/ticket/13515 https://svn.boost.org/trac10/ticket/13515 <ul> <li><strong>attachment</strong> → <span class="trac-field-new">asio_bug_sample.tar.bz2</span> </li> </ul> Ticket anonymous Tue, 31 Jul 2018 14:18:58 GMT <link>https://svn.boost.org/trac10/ticket/13515#comment:1 </link> <guid isPermaLink="false">https://svn.boost.org/trac10/ticket/13515#comment:1</guid> <description> <p> Hello, </p> <pre class="wiki">while (read_ec != boost::asio::error::eof); </pre><p> error::eof is an enum value of type boost::asio::error::errc; in this context read_ec will be converted to bool and compared against the integer value of error::eof. </p> <p> Try to compare against <code>read_ec.value()</code> instead. </p> </description> <category>Ticket</category> </item> <item> <dc:creator>bronf</dc:creator> <pubDate>Wed, 01 Aug 2018 19:59:31 GMT</pubDate> <title/> <link>https://svn.boost.org/trac10/ticket/13515#comment:2 </link> <guid isPermaLink="false">https://svn.boost.org/trac10/ticket/13515#comment:2</guid> <description> <blockquote class="citation"> <p> error::eof is an enum value of type boost::asio::error::errc; in this context read_ec will be converted to bool and compared against the integer value of error::eof. </p> </blockquote> <p> Apparently this not the case. I tested in gdb and boost::asio::error::eof is first converted to an error_code which is then compared to read_ec. </p> </description> <category>Ticket</category> </item> </channel> </rss>