id	summary	reporter	owner	description	type	status	milestone	component	version	severity	resolution	keywords	cc
9105	boost::tokenizer fails if string contains percent signs	anonymous	jsiek	"[Visual Studio 2010]
Let's use as an example code shown on the bottom of the page http://www.boost.org/doc/libs/1_54_0/libs/tokenizer/escaped_list_separator.htm

=====>>>>
// simple_example_2.cpp
#include<iostream>
#include<boost/tokenizer.hpp>
#include<string>

int main(){
   using namespace std;
   using namespace boost;
   string s = ""Field 1,\""putting quotes around fields, allows commas\"",Field 3"";
   tokenizer<escaped_list_separator<char> > tok(s);
   for(tokenizer<escaped_list_separator<char> >::iterator beg=tok.begin(); beg!=tok.end();++beg){
       cout << *beg << ""\n"";
   }
}
<<<<=====

Works fine.

Let's insert percent sign somewhere inside string s.
As an example
s = ""Field 1,\""putting q%uotes around fields, allows commas\"",Field 3"";

program fails with:
First-chance exception at 0x75c4c41f in test.exe: Microsoft C++ exception: boost::escaped_list_error at memory location 0x007cf060..

It fails even percent sign would be followed by hex code (like %20)
"	Bugs	new	To Be Determined	tokenizer	Boost 1.54.0	Problem