Boost C++ Libraries: Ticket #8577: Multiple qi::match calls on the same stream corrupt input data https://svn.boost.org/trac10/ticket/8577 <p> qi::match cannot be reliably called multiple times on the same input stream. Probably it consumes some characters from the stream to do look-ahead but doesn't return them back to the stream on destruction. So it only works reliably if it's the last extraction operator for a given stream. </p> <p> I know there may be problems to return more than one character to a stream, so if it's the expected qi::match behavior, it's worth to be mentioned in the documentation. </p> <p> The following code illustrates the problem: </p> <pre class="wiki">#include &lt;boost/spirit/include/qi.hpp&gt; #include &lt;boost/spirit/include/qi_match.hpp&gt; #include &lt;iostream&gt; #include &lt;sstream&gt; namespace qi = boost::spirit::qi; int main() { std::istringstream is("11 22 33 44 55 "); is &gt;&gt; std::noskipws; int n = 0; for (int i = 0; i &lt; 5; ++i) { is &gt;&gt; qi::match(qi::int_ &gt;&gt; +qi::lit(' '), n); assert(is); std::cout &lt;&lt; n &lt;&lt; ' '; } } </pre><pre class="wiki">Expected output: 11 22 33 44 55 Actual output: 11 2 3 4 5 </pre> en-us Boost C++ Libraries /htdocs/site/boost.png https://svn.boost.org/trac10/ticket/8577 Trac 1.4.3 anonymous Sat, 16 Dec 2017 23:29:08 GMT <link>https://svn.boost.org/trac10/ticket/8577#comment:1 </link> <guid isPermaLink="false">https://svn.boost.org/trac10/ticket/8577#comment:1</guid> <description> <blockquote class="citation"> <p> Probably it consumes some characters from the stream to do look-ahead </p> </blockquote> <p> It will consume data even if your parser is LL(1). </p> <blockquote class="citation"> <p> but doesn't return them back to the stream on destruction </p> </blockquote> <p> You cannot return read data back to input stream (read about <a class="missing wiki">InputIterator</a> concept). </p> <blockquote class="citation"> <p> so if it's the expected qi::match behavior, it's worth to be mentioned in the documentation. </p> </blockquote> <p> It should be clear to everyone that to parse data from an input stream you must pull data from the stream and you cannot place the read data back to it, even a single character. </p> <hr /> <p> All <code>qi::match</code> is doing is wrapping your iterators with <code>spirit::basic_istream_iterator</code>, calling <code>qi::parse</code> and setting <code>failbit</code> if parsing failed. You should not use <code>qi::match</code> in loop on the same stream at least because it is slow (<code>spirit::basic_istream_iterator</code> lots of heap allocations). Only use <code>qi::match</code> to match all of your input. </p> <p> Hint: Replace <code>+qi::lit(' ')</code> with a skipper (i.e. use <code>qi::phrase_match</code>) </p> </description> <category>Ticket</category> </item> <item> <author>Nikita Kniazev <nok.raven@…></author> <pubDate>Sat, 16 Dec 2017 23:29:28 GMT</pubDate> <title/> <link>https://svn.boost.org/trac10/ticket/8577#comment:2 </link> <guid isPermaLink="false">https://svn.boost.org/trac10/ticket/8577#comment:2</guid> <description> <blockquote class="citation"> <p> Probably it consumes some characters from the stream to do look-ahead </p> </blockquote> <p> It will consume data even if your parser is LL(1). </p> <blockquote class="citation"> <p> but doesn't return them back to the stream on destruction </p> </blockquote> <p> You cannot return read data back to input stream (read about <a class="missing wiki">InputIterator</a> concept). </p> <blockquote class="citation"> <p> so if it's the expected qi::match behavior, it's worth to be mentioned in the documentation. </p> </blockquote> <p> It should be clear to everyone that to parse data from an input stream you must pull data from the stream and you cannot place the read data back to it, even a single character. </p> <hr /> <p> All <code>qi::match</code> is doing is wrapping your iterators with <code>spirit::basic_istream_iterator</code>, calling <code>qi::parse</code> and setting <code>failbit</code> if parsing failed. You should not use <code>qi::match</code> in loop on the same stream at least because it is slow (<code>spirit::basic_istream_iterator</code> lots of heap allocations). Only use <code>qi::match</code> to match all of your input. </p> <p> Hint: Replace <code>+qi::lit(' ')</code> with a skipper (i.e. use <code>qi::phrase_match</code>) </p> </description> <category>Ticket</category> </item> <item> <dc:creator>Joel de Guzman</dc:creator> <pubDate>Sun, 17 Dec 2017 23:46:48 GMT</pubDate> <title>status changed; resolution set https://svn.boost.org/trac10/ticket/8577#comment:3 https://svn.boost.org/trac10/ticket/8577#comment:3 <ul> <li><strong>status</strong> <span class="trac-field-old">new</span> → <span class="trac-field-new">closed</span> </li> <li><strong>resolution</strong> → <span class="trac-field-new">invalid</span> </li> </ul> Ticket