Boost C++ Libraries: Ticket #1836: bug in serializing wide character strings

Fri, 23 May 2008 12:40:13 GMT

Why don't you use text_wi|oarchive? This was designed for wide characters and strings.

jefffaust — Fri, 23 May 2008 14:23:32 GMT

Replying to Ostap Kutsyy <ostapkl@gmail.com>:

Why don't you use text_wi|oarchive? This was designed for wide characters and strings.

Frankly, I've never seen those in the documentation. Now that I look, there they are... in the "Archive Concepts" and "Implementation Notes" sections. The wide character classes are not even part of the "Text Archive Class Diagram".

I've asked the developer working on this if this will solve our problem. I'll follow up after he looks into it.

Thanks for the help!

Jeff

jefffaust — Fri, 23 May 2008 17:48:30 GMT

Ostap,

Using text_w?archive does address our problem. Thanks for the help. However, I still consider this a bug. Attempting to serialize a wstring should fail to compile, in the same way that "cout << wstring();" fails to compile. As it is currently, it fails at runtime in frustratingly subtle ways.

Jeff

status changed

Robert Ramey — Mon, 09 Jun 2008 19:16:14 GMT

status new → assigned

milestone deleted

Mon, 03 Nov 2008 14:20:34 GMT

milestone Boost 1.35.1

Milestone Boost 1.35.1 deleted

cc changed

Sohail Somani — Sat, 30 May 2009 04:02:39 GMT

cc Sohail Somani added

Robert, do you have any thoughts on this? My thoughts are that narrow archives should definitely not support wide character streams for this reason (among others...)

If you agree, I have a patch in the works for this issue.

Sohail Somani — Sat, 30 May 2009 04:03:12 GMT

Replying to sohail:

Robert, do you have any thoughts on this? My thoughts are that narrow archives should definitely not support wide character streams for this reason (among others...)

Should not support wide characters...

Robert Ramey — Sat, 30 May 2009 04:59:13 GMT

I don't think that data types should be coupled to archives.

That is, any data that can be serialized to one kind of archive should be serializable to ALL kinds of archives. That is why std::wstring must be serializable into a text or xml_archive. The real fix is to make adjustments so that all characters are rendered. This is not trivial to do without a big performance hit. So its and open issue for now.

milestone set

Dean Michael Berris — Mon, 29 Nov 2010 02:41:35 GMT

milestone → To Be Determined

Replying to ramey:

This is not trivial to do without a big performance hit. So its and open issue for now.

Do you have a suggestion as to how this should be addressed? For example, should serializing a std::wstring to a text archive yield an appropriately encoded (maybe Base64) string, then when read back be appropriately decoded? How will this work on binary archives and in other user-provided archives (like the ones Boost.MPI provides)?

Robert Ramey — Mon, 29 Nov 2010 20:32:54 GMT

speaking from memory when I last looked at this, I concluded that what was needed was an escape mechanism. This would render some characters as \134 or something like that. This would be implemented in the "stack" of iterator adaptors which are used to save/load the string. This would entail character by character processing which I was thinking would be a performance hit - which it would be. BUT, now I realize that serialization of a wstring to a char archive is not a common operation, so performance really isn't an issue. The way to address this is to look at the documentation for "dataflow" iterators and the related code. It wouldn't be too hard to craft another "escape" layer and insert into the iterator stack which handles this. Feel free to take this on.

Robert Ramey