id	summary	reporter	owner	description	type	status	milestone	component	version	severity	resolution	keywords	cc
6024	Behaviour and documentation don't match	Olaf van der Spek <olafvdspek@…>	jsiek	"Hi,

> explicit char_separator()

Explicit? Doesn't explicit only apply to one argument constructors?

> The function std::isspace() is used to identify dropped delimiters and std::ispunct() is used to identify kept delimiters. In addition, empty tokens are dropped.

 std::string s(""[1][3][5][]9"");
 BOOST_FOREACH(auto i, boost::tokenizer<>(s))
   std::cout << i << std::endl;

This outputs only numbers. Shouldn't the brackets be included too?"	Bugs	new	To Be Determined	tokenizer	Boost 1.47.0	Problem