Opened 8 years ago
Last modified 8 years ago
#10022 new Feature Requests
json_parser_write inconditionally converts CP1252 to escaped unicode
Reported by: | anonymous | Owned by: | Sebastian Redl |
---|---|---|---|
Milestone: | To Be Determined | Component: | property_tree |
Version: | Boost 1.55.0 | Severity: | Problem |
Keywords: | UTF-8 | Cc: |
Description
See function 'create_escapes' in property_tree/detail/json_parser_write.hpp. I supply a property_tree containing UTF-8 encoded values, encoded in a custom way from data in different code pages. I want JSON with real UTF-8 encoding, which the JSON standard just allows (as far as I googled). So here the encoding of 'special' characters to "\uXXXX" should be turned off.
All values the Ch has are directly translated to a unicode number, but when Ch=char this only works correct when the char data is in code page 1252.
Currently the option to encode the characters nor the used code page are parameters here.
I needed to locally adapt the code like in http://stackoverflow.com/questions/10260688/boostproperty-treejson-parser-and-two-byte-wide-characters
Now comment in the code states: "This assumes an ASCII superset. But so does everything in PTree."
A PTree is, as far as used to serialize / parse JSON, fine with handling UTF-8 encoded values.
And: "We escape everything outside ASCII, because this code can't handle high unicode characters."
Strange, the JSON generated is parsed fine using boost and the resulting PTree contains correctly encoded values again.