Boost C++ Libraries: Ticket #469: multitoken broken

nobody — Mon, 19 Mar 2007 20:48:18 GMT

Logged In: NO
I have encounter the same problems. I need input multiple multitoken. The program options 1.33 does not work in the mutitoken, if it use in the middle of command line. Does anyone have fixed this bug?

summary changed; severity, milestone set

Vladimir Prus — Wed, 11 Jul 2007 18:37:13 GMT

summary multitoken broken in program_options 1.33 → multitoken broken
severity → Problem
milestone → Boost 1.35.0

Confirmed as a problem. The solution to be decided.

anonymous — Thu, 26 Jul 2007 21:01:14 GMT

Given the following program_options description:

std::vector<std::string> multiItems;
program_description desc;
desc.add_options()
    ("multi", value<>(&multiItems)->multitoken(), "...");
....

Would we expect the command line: "myProgram --multi foo bar" to add two entries (foo and bar) into 'multiItems'?

It currently (1.34) doesn't; bar gets lost. I can understand that this could be tricky, especially if positional options are also being used.

If multitoken was never designed for this purpose, then currently the way to achieve the desired effect is to overload 'validate'. However, this is only necessary because of the use of 'lexical_cast' in the current validate implementations.

To illustrate:

typedef std::pair<int int> PairOfInts;
PairOfInts pairOfInts;
program_description desc;
desc.add_options()
    ("pair", value<>(&pairOfInts), "...");
....

With command line: "myProgram --pair 1 2":

The current 'validate' code will stream (via lexical_cast) the string "1 2" into lexical_stream, then stream it out to the value type, i.e. the std::pair.

The problem there is that the stream-out of the string will only take the "1" part, so when it attempts to stream it back in, only one value is presented.

If we had provided a stream operator for our pair:

std::istream& operator>>(std::istream &a_istr, PairOfInts &a_pair)
{
    a_istr >> a_pair.first >> a_pair.second;
    return a_istr;
}

Then an exception would fire trying to get the second value.

The 'validate' code has a vector of strings, but only the first entry is filled.

If the current (detail/value_semantic.hpp) validate code:

template<class T, class charT>
void validate(boost::any& v,
              const std::vector< std::basic_string<charT> >& xs,
              T*, long)
{
    validators::check_first_occurrence(v);
    std::basic_string<charT> s(validators::get_single_string(xs));
    try {
        v = any(lexical_cast<T>(s));
    }
    catch(const bad_lexical_cast&) {
        boost::throw_exception(invalid_option_value(s));
    }
}

Was adjusted to not use lexical_cast, e.g.:

template<class T, class charT>
void validate(boost::any& v,
              const std::vector< std::basic_string<charT> >& xs,
              T*, long)
{
    validators::check_first_occurrence(v);
    std::basic_string<charT> s(validators::get_single_string(xs));
    std::istringstream istr(s);
    T value;
    if (!(istr >> value)
    {
        boost::throw_exception(invalid_option_value(s));
    }
    v = value;
}

Then more complex types could be supported just by providing the stream-in operator, rather than overloaded 'validate's (that seem to have to go in namespace boost, too).

status, version, component, severity, milestone changed

anonymous — Thu, 12 Mar 2009 13:34:00 GMT

status assigned → new
version None → Boost 1.38.0
component config → program_options
severity Problem → Showstopper
milestone Boost 1.36.0 → To Be Determined

This multitoken bug has been broken and reported as a bug for 4 years now and not addressed. Is the author/supported not planning on ever fixing it? I'm stuck on boost 1.32 just because of this one bug.

status, resolution changed

Vladimir Prus — Fri, 03 Apr 2009 13:42:33 GMT

status new → closed
resolution None → fixed

(In [52154]) When processing value multitoken options, don't eat futher options. Fixes #469.

Vladimir Prus — Mon, 06 Apr 2009 09:36:27 GMT

(In [52210]) Merge from release: When processing value multitoken options, don't eat futher options. Fixes #469.