Multiple qi::match calls on the same stream corrupt input data

[Vadim Guchenko <yhw@…>]

qi::match cannot be reliably called multiple times on the same input stream. Probably it consumes some characters from the stream to do look-ahead but doesn't return them back to the stream on destruction. So it only works reliably if it's the last extraction operator for a given stream.

I know there may be problems to return more than one character to a stream, so if it's the expected qi::match behavior, it's worth to be mentioned in the documentation.

The following code illustrates the problem:

#include <boost/spirit/include/qi.hpp>
#include <boost/spirit/include/qi_match.hpp>
#include <iostream>
#include <sstream>

namespace qi = boost::spirit::qi;

int main() {
    std::istringstream is("11 22 33 44 55 ");
    is >> std::noskipws;
    int n = 0;

    for (int i = 0; i < 5; ++i) {
        is >> qi::match(qi::int_ >> +qi::lit(' '), n);
        assert(is);
        std::cout << n << ' ';
    }
}

Expected output: 11 22 33 44 55 
Actual output:   11 2 3 4 5 

[anonymous]

Probably it consumes some characters from the stream to do look-ahead

It will consume data even if your parser is LL(1).

but doesn't return them back to the stream on destruction

You cannot return read data back to input stream (read about InputIterator concept).

so if it's the expected qi::match behavior, it's worth to be mentioned in the documentation.

It should be clear to everyone that to parse data from an input stream you must pull data from the stream and you cannot place the read data back to it, even a single character.

---

All qi::match is doing is wrapping your iterators with spirit::basic_istream_iterator, calling qi::parse and setting failbit if parsing failed. You should not use qi::match in loop on the same stream at least because it is slow (spirit::basic_istream_iterator lots of heap allocations). Only use qi::match to match all of your input.

Hint: Replace +qi::lit(' ') with a skipper (i.e. use qi::phrase_match)

[Nikita Kniazev <nok.raven@…>]

Probably it consumes some characters from the stream to do look-ahead

It will consume data even if your parser is LL(1).

but doesn't return them back to the stream on destruction

You cannot return read data back to input stream (read about InputIterator concept).

so if it's the expected qi::match behavior, it's worth to be mentioned in the documentation.

It should be clear to everyone that to parse data from an input stream you must pull data from the stream and you cannot place the read data back to it, even a single character.

---

All qi::match is doing is wrapping your iterators with spirit::basic_istream_iterator, calling qi::parse and setting failbit if parsing failed. You should not use qi::match in loop on the same stream at least because it is slow (spirit::basic_istream_iterator lots of heap allocations). Only use qi::match to match all of your input.

Hint: Replace +qi::lit(' ') with a skipper (i.e. use qi::phrase_match)