id,summary,reporter,owner,description,type,status,milestone,component,version,severity,resolution,keywords,cc
5033,Property Tree JSON Parser cannot handle utf-8 string correctly.,Lorin Liu <liu.lorin@…>,Sebastian Redl,"Please refer to the following code fragment.

{{{
    // This is a json utf-8 string {""value"": ""天津""}
    const char json[] = {0x7B, 0x22, 0x76, 0x61, 0x6C, 0x75, 0x65, 0x22, 0x3A, 0x20, 
                         0x22, 0xE5, 0xA4, 0xA9, 0xE6, 0xB4, 0xA5, 0x22, 0x7D, 0x00};

    boost::property_tree::ptree pt;
    boost::format fmter(""%1% : %2% \n"");
    std::stringstream strm;
    std::string value;

    strm << json;

    read_json(strm, pt);

    value = pt.get<std::string>(""value"");

    // Print the individual char one by one.
    // However the wrong result appears. All chars printed to console are 0x7F. 
    // And the expected result should be the chars of 0xE5, 0xA4, 0xA9, 0xE6, 0xB4 and 0xA5.
    BOOST_FOREACH(char c, value)
        std::cout << (fmter % (int)(unsigned char) c % c) << std::endl;
}}}

After my investigation, this might be a bug in boost/property_tree/detail/json_parser_read.hpp.

My patch for this issue is as follows.

{{{
--- json_parser_read.hpp.orig	2010-12-24 15:49:06.000000000 +0800
+++ json_parser_read.hpp	2011-01-02 10:26:37.000000000 +0800
@@ -145,7 +145,7 @@
             a_unicode(context &c): c(c) { }
             void operator()(unsigned long u) const
             {
-                u = (std::min)(u, static_cast<unsigned long>((std::numeric_limits<Ch>::max)()));
+                // u = (std::min)(u, static_cast<unsigned long>((std::numeric_limits<Ch>::max)()));
                 c.string += Ch(u);
             }
         };
}}}
",Bugs,closed,To Be Determined,property_tree,Boost 1.45.0,Problem,fixed,,