multitoken broken

[rutavibaja]

Due to the changes in the parser, multitoken is broken 
in program_options [boost 1.33]. Looking at the code it 
is relatively clear why - the parser is too "greedy" and 
eats up all the options following the multitoken one 
without checking when to stop (for example, it should 
stop once it encounters another option/switch).

If this functionality is to be deprecated as hinted in 
some mails on the mailing list, it should be removed 
from the public interface. If not, I guess it should be 
fixed :)

The way to test it is to have a multitoken switch appear 
_in the middle_ of the command line (definitely not at 
the end) which should then simply eat all options to the 
right of it.

[nobody]

Logged In: NO 

I have encounter the same problems. I need input multiple multitoken. The program options 1.33 does not work in the mutitoken, if it use in the middle of command line. Does anyone have fixed this bug?

[Vladimir Prus]

Confirmed as a problem. The solution to be decided.

[anonymous]

Given the following program_options description:

std::vector<std::string> multiItems;

program_description desc;
desc.add_options()
    ("multi", value<>(&multiItems)->multitoken(), "...");
....

Would we expect the command line: "myProgram --multi foo bar" to add two entries (foo and bar) into 'multiItems'?

It currently (1.34) doesn't; bar gets lost. I can understand that this could be tricky, especially if positional options are also being used.

If multitoken was never designed for this purpose, then currently the way to achieve the desired effect is to overload 'validate'. However, this is only necessary because of the use of 'lexical_cast' in the current validate implementations.

To illustrate:

typedef std::pair<int int> PairOfInts;
PairOfInts pairOfInts;

program_description desc;
desc.add_options()
    ("pair", value<>(&pairOfInts), "...");
....

With command line: "myProgram --pair 1 2":

The current 'validate' code will stream (via lexical_cast) the string "1 2" into lexical_stream, then stream it out to the value type, i.e. the std::pair.

The problem there is that the stream-out of the string will only take the "1" part, so when it attempts to stream it back in, only one value is presented.

If we had provided a stream operator for our pair:

std::istream& operator>>(std::istream &a_istr, PairOfInts &a_pair)
{
    a_istr >> a_pair.first >> a_pair.second;
    return a_istr;
}

Then an exception would fire trying to get the second value.

The 'validate' code has a vector of strings, but only the first entry is filled.

If the current (detail/value_semantic.hpp) validate code:

template<class T, class charT>
void validate(boost::any& v, 
              const std::vector< std::basic_string<charT> >& xs, 
              T*, long)
{
    validators::check_first_occurrence(v);
    std::basic_string<charT> s(validators::get_single_string(xs));
    try {
        v = any(lexical_cast<T>(s));
    }
    catch(const bad_lexical_cast&) {
        boost::throw_exception(invalid_option_value(s));
    }
}

Was adjusted to not use lexical_cast, e.g.:

template<class T, class charT>
void validate(boost::any& v, 
              const std::vector< std::basic_string<charT> >& xs, 
              T*, long)
{
    validators::check_first_occurrence(v);
    std::basic_string<charT> s(validators::get_single_string(xs));
    std::istringstream istr(s);
    T value;
    if (!(istr >> value)
    {
        boost::throw_exception(invalid_option_value(s));
    }
    v = value;
}

Then more complex types could be supported just by providing the stream-in operator, rather than overloaded 'validate's (that seem to have to go in namespace boost, too).

[anonymous]

This multitoken bug has been broken and reported as a bug for 4 years now and not addressed. Is the author/supported not planning on ever fixing it? I'm stuck on boost 1.32 just because of this one bug.

[Vladimir Prus]

(In [52154]) When processing value multitoken options, don't eat futher options. Fixes #469.

[Vladimir Prus]

(In [52210]) Merge from release: When processing value multitoken options, don't eat futher options. Fixes #469.