Boost C++ Libraries: Ticket #11327: boost::wave produces non-existent tokens

status changed; resolution set

Hartmut Kaiser — Thu, 21 May 2015 01:48:18 GMT

status new → closed
resolution → invalid

This is on purpose as it reflects the token stream which would be generated by a preprocessor. If you don't want to see the #line directives generated, add:

      WaveContext.set_language(
          boost::wave::enable_emit_line_directives(
              ctx.get_language(),
              false));

before iterating over the input (see here for more information: http://www.boost.org/doc/libs/1_58_0/libs/wave/doc/class_reference_context.html).

Vyacheslav Andrejev — Thu, 21 May 2015 19:58:20 GMT

Thanks!

Vyacheslav Andrejev — Thu, 21 May 2015 22:54:17 GMT

Actually enable_emit_line_directives has undesired effect: the lexer not only stops producing not existing #line, but also removes existing. For example

#include <iostream>
#include <string>
#include <boost/wave.hpp>
#include <boost/wave/cpplexer/cpp_lex_iterator.hpp>
using namespace boost::wave;
using namespace boost::wave::cpplexer;
int main(int argc, char* argv[])
{
    std::string str = "int a;\n#line 10 \"file.cpp\"\nint b;\n";
    typedef lex_iterator<lex_token<> > LexIterator;
    typedef boost::wave::context<std::string::const_iterator, LexIterator> WaveContext;
    WaveContext ctx(str.cbegin(), str.cend());
    ctx.set_language(
        boost::wave::enable_emit_line_directives(
        ctx.get_language(),
        false));
    for (auto tok : ctx)
    {
        std::cout << get_token_name(token_id(tok)) << '('
                  << tok.get_value() << ')' << std::endl;
    }
    return 0;
}

The output:

INT(int)
SPACE( )
IDENTIFIER(a)
SEMICOLON(;)
NEWLINE(
)
INT(int)
SPACE( )
IDENTIFIER(b)
SEMICOLON(;)
NEWLINE(
)
EOF()

Is there a way to leave just existing #line?

Hartmut Kaiser — Fri, 22 May 2015 10:00:52 GMT

Wave always consumes any #line directives in the input to correctly assign line numbers to the generated tokens (like it consumes all other preprocessor directives). It outputs those #line directives based on its own heuristics. There is currently no way to retain those from the input.

Vyacheslav Andrejev — Wed, 27 May 2015 21:10:40 GMT

It produces incorrect line numbers if a file already has #line directives.