id	summary	reporter	owner	description	type	status	milestone	component	version	severity	resolution	keywords	cc
293	tokenizer and signed char	nobody	Markus Schöpflin	"{{{
boost::tokenizer incorrect works on platforms on which 
plain char is signed. For example, at least in 
boost::char_delimiters_separator::is_nonret should be 
explicit conversion from char type to int type. 
Something like this std::isspace
(std::char_traits<Char>::to_int_type(E)). Minimal 
reproducer for this issue which uses Russian letters 
results in assert in MSVC 7.1:

// cl /Zi /MDd -D_DEBUG test.cpp
#include <boost\tokenizer.hpp>
#include <string>
#include <iostream>
using namespace std;
using namespace boost;  
  
int main()
{
  string s = ""4@02AB2C9 <8@!"";
    
  tokenizer<> tok(s);
  for (tokenizer<>::iterator beg=tok.begin(); beg!
=tok.end();++beg)
  {
    cout << *beg << ""\n"";
  }
}


}}}"	Bugs	closed		None	None		None