id,summary,reporter,owner,description,type,status,milestone,component,version,severity,resolution,keywords,cc
6024,Behaviour and documentation don't match,Olaf van der Spek <olafvdspek@…>,jsiek,"Hi,

> explicit char_separator()

Explicit? Doesn't explicit only apply to one argument constructors?

> The function std::isspace() is used to identify dropped delimiters and std::ispunct() is used to identify kept delimiters. In addition, empty tokens are dropped.

 std::string s(""[1][3][5][]9"");
 BOOST_FOREACH(auto i, boost::tokenizer<>(s))
   std::cout << i << std::endl;

This outputs only numbers. Shouldn't the brackets be included too?",Bugs,new,To Be Determined,tokenizer,Boost 1.47.0,Problem,,,