XML Serialization error when loading class that inherits from wstring

[nitram.cero@…]

Using version 1.37.0, but checked against SVN and it doesn't seem to have changed a lot. I'm sorry and I'll try the latest boost version, but please do not disregard this bug just because of the version.

I've tested up until I reach basic_text_iprimitive::load(wstring &t):

    template<class T>
    void load(T & t)
    {
        if(is.fail())
            boost::serialization::throw_exception(
                archive_exception(archive_exception::stream_error)
            );
        is >> t;
    }

The value was "./" In this case t = "./</basePath>" :-O

(basePath thas the NVP name for the wstring value)

As the end tag is taken as the actual value, trying to get the end tag will break the serialization when continuing the execution.

Call stack:

 	CharacterEditor.exe!boost::archive::basic_text_iprimitive<std::basic_istream<wchar_t,std::char_traits<wchar_t> > >::load<nt::wstring>(nt::wstring & t={...})  Line 87	C++
 	CharacterEditor.exe!boost::archive::xml_wiarchive_impl<boost::archive::xml_wiarchive>::load<nt::wstring>(nt::wstring & t={...})  Line 63	C++
 	CharacterEditor.exe!boost::archive::load_access::load_primitive<boost::archive::xml_wiarchive,nt::wstring>(boost::archive::xml_wiarchive & ar={...}, nt::wstring & t={...})  Line 98	C++
 	CharacterEditor.exe!boost::archive::detail::load_non_pointer_type<boost::archive::xml_wiarchive,nt::wstring>::load_primitive::invoke(boost::archive::xml_wiarchive & ar={...}, nt::wstring & t={...})  Line 306 + 0xd bytes	C++
 	CharacterEditor.exe!boost::archive::detail::load_non_pointer_type<boost::archive::xml_wiarchive,nt::wstring>::invoke(boost::archive::xml_wiarchive & ar={...}, nt::wstring & t={...})  Line 391 + 0xd bytes	C++
 	CharacterEditor.exe!boost::archive::load<boost::archive::xml_wiarchive,nt::wstring>(boost::archive::xml_wiarchive & ar={...}, nt::wstring & t={...})  Line 514 + 0xd bytes	C++
 	CharacterEditor.exe!boost::archive::detail::common_iarchive<boost::archive::xml_wiarchive>::load_override<nt::wstring>(nt::wstring & t={...}, int __formal=0)  Line 59 + 0x15 bytes	C++
 	CharacterEditor.exe!boost::archive::basic_xml_iarchive<boost::archive::xml_wiarchive>::load_override<nt::wstring>(const boost::serialization::nvp<nt::wstring> & t={...}, int __formal=0)  Line 82	C++
>	CharacterEditor.exe!boost::archive::xml_wiarchive_impl<boost::archive::xml_wiarchive>::load_override<boost::serialization::nvp<nt::wstring> const >(const boost::serialization::nvp<nt::wstring> & t={...}, int __formal=0)  Line 79	C++
.
.
.
.

nt::string is almost the same as std::wstring

namespace nt { class wstring : public std::wstring { /*some extra helper functions for initialization from a wxWidgets' wxString*/} }
BOOST_CLASS_IMPLEMENTATION(nt::wstring, boost::serialization::primitive_type)

namespace boost { namespace serialization {

template<class Archive>
void serialize(Archive & ar, nt::wstring &s, const unsigned int version)
{
	ar & static_cast<std::wstring &>(s);
}

} } // namespace serialization // namespace boost

The compiler is Microsoft Visual C++ Express 2008 (9) I did this hacks in order to get serialization templates to work under MSVC9:

//HACK: enable serialization of MSVC9 hash_map
#define BOOST_HAS_HASH
#define BOOST_HASH_MAP_HEADER		<hash_map>

//HACK: disable resize of hash_map
#define __MWERKS__
#include <boost/serialization/hash_collections_load_imp.hpp>
#undef __MWERKS__

#include <boost/serialization/hash_map.hpp>

Thanks for your time. -Martín

[nitram.cero@…]

The same happens with SVN version (checked)

and also by doing this (that seems more proper):

template<class Archive>
void serialize(Archive & ar, nt::wstring &s, const unsigned int version)
{
	ar & boost::serialization::base_object<std::wstring>(s);
}

Regards -Martín

[nitram.cero@…]

Ok, I've been investigating a bit more.

Serialization in XML of nt::wstring (my std::wstring derived class) as a primitive makes it get loaded by a template instead of an specialized std::wstring function. And that reads until a space (sometimes including the end tag).

If I don't define my nt::wstring class as a primitive type, and wrap the base (std::wstring) as a NVP, then I get a redundant tag (i.e: <my_value><base>the string</base></my_value>)

Shouldn't be a wrapper to explicitly define that it's unnamed and shouldn't produce a tags?

or maybe a having a "primitive_string_type" instead of "primitive_type" that has std::wstring and std::string specialized.

BOOST_CLASS_IMPLEMENTATION(nt::wstring, boost::serialization::primitive_type)

Fails, loads as with a generic template instead of std::wstring specific

namespace boost { namespace serialization {

template<class Archive>
void serialize(Archive & ar, nt::wstring &s, const unsigned int version)
{
	ar & boost::serialization::make_nvp("base", static_cast<std::wstring &>(s));
}

} } // namespace serialization // namespace boost

This works but for each string I have a redundant tag just for the base.

For now I'll have it wrapped around "<base>" tags. I don't know if this trac entry should be of type Bugs or Feature request... I'll just leave it this way, feel free to change it.

Regards -Martín

[Robert Ramey]

usually this occurs when the saving archive is closed before the saving stream is flushed. Could you enclose a small example along with an xml archive which demonstrates this problem?

[anonymous]

There is no problem saving, but loading. The output XML looks fine.

But when loading a derived from std::wstring/string class, the loading template specialization is not contemplated, and the operator>> on the stream is called as if it were an int, float, any non-specialized type.

I.e.: Instead of calling the loading function for the parameter wstring&, it calls the generic one.

There could be a class implementation that was primitive_string, besides primitive, to allow for customized std::string/wstring

Regards -Martín

[Robert Ramey]

How about attaching a small test program which shows the failure?

[nitram.cero@…]

No problem. (Using MSVC++ 2008 express)

Important excerpts:

namespace primitive {
	class wstring : public std::wstring {


...



namespace object {
	class wstring : public std::wstring {


...


BOOST_CLASS_IMPLEMENTATION(object::wstring, boost::serialization::object_serializable)
BOOST_CLASS_IMPLEMENTATION(primitive::wstring, boost::serialization::primitive_type)


...

template<class Archive>
void serialize(Archive & ar, object::wstring &s, const unsigned int version)
{
	ar & boost::serialization::make_nvp("base", static_cast<std::wstring &>(s));
	//ar & boost::serialization::base_object<std::wstring>(s);
}

Code:

#define _CRT_SECURE_NO_WARNINGS

#include <boost/archive/xml_woarchive.hpp>
#include <boost/archive/xml_wiarchive.hpp>
#include <boost/archive/xml_archive_exception.hpp>

#include <boost/serialization/string.hpp>


#include <string>
#include <fstream>
#include <iostream>



//Utility defines for better readability
#define SERIALIZE_FUNCTION																	\
    friend class boost::serialization::access;												\
    template<class Archive>																	\
    void serialize(Archive & serializable_archive, const unsigned int serializable_version)

#define serializable_base(BASECLASS)		boost::serialization::make_nvp(#BASECLASS, boost::serialization::base_object< BASECLASS >(*this))
#define serializable_variable(VAR)			boost::serialization::make_nvp(#VAR, VAR)


namespace primitive {

	class wstring : public std::wstring {
	public:
		wstring()																{ }
		wstring(const wchar_t *str)				: std::wstring(str)				{ }
		wstring(const wstring &wstr)			: std::wstring(wstr)			{ }
		wstring(const std::wstring &wstr)		: std::wstring(wstr)			{ }
		
		wstring &operator =(const wstring &wstr)			{ operator =(static_cast<const std::wstring &>(wstr));	return *this; }
		wstring &operator =(const std::wstring &wstr)		{ std::wstring::operator =(wstr);						return *this; }
		wstring &operator =(const wchar_t *str)				{ std::wstring::operator =(str);						return *this; }

		//...custom stuff...
	};

}

namespace object {

	class wstring : public std::wstring {
	public:
		wstring()																{ }
		wstring(const wchar_t *str)				: std::wstring(str)				{ }
		wstring(const wstring &wstr)			: std::wstring(wstr)			{ }
		wstring(const std::wstring &wstr)		: std::wstring(wstr)			{ }
		
		wstring &operator =(const wstring &wstr)			{ operator =(static_cast<const std::wstring &>(wstr));	return *this; }
		wstring &operator =(const std::wstring &wstr)		{ std::wstring::operator =(wstr);						return *this; }
		wstring &operator =(const wchar_t *str)				{ std::wstring::operator =(str);						return *this; }

		//...custom stuff...
	};

}

//BOOST_CLASS_IMPLEMENTATION(nt::wstring, boost::serialization::primitive_type)
BOOST_CLASS_IMPLEMENTATION(object::wstring, boost::serialization::object_serializable)
BOOST_CLASS_IMPLEMENTATION(primitive::wstring, boost::serialization::primitive_type)

namespace boost { namespace serialization {

template<class Archive>
void serialize(Archive & ar, object::wstring &s, const unsigned int version)
{
	ar & boost::serialization::make_nvp("base", static_cast<std::wstring &>(s));
	//ar & boost::serialization::base_object<std::wstring>(s);
}

} } // namespace serialization // namespace boost


using std::wcout;
using std::endl;

template <typename T>
void do_stuff(char *filename, const T &value)
{
	try {
		//save
		{
			T std_wstr = value;
			wcout << "saving: " << std_wstr << endl;
			
			std::wofstream os(filename);
			boost::archive::xml_woarchive oxml(os);
			
			oxml & serializable_variable(std_wstr);
		}
		//load
		{
			T std_wstr;
			
			std::wifstream os(filename);
			boost::archive::xml_wiarchive oxml(os);
			
			oxml & serializable_variable(std_wstr);
			wcout << "loaded: " << std_wstr << " " << (std_wstr == value ? "(EQUAL)" : "(NOT EQUAL)") << endl;
		}		
		
		wcout << "*** Success" << endl;
	}
	catch(boost::archive::xml_archive_exception &e) {
		wcout << "*** Archive failure: " << e.what() << "***" << endl;
	}
	catch(std::exception &e) {
		wcout << "*** Failure: " << e.what() << "***" << endl;
	}
}


int main(int argc, char **argv)
{	
	wcout << L"---------- std::wstring ----------" << endl;
	//"official" wstring	[OK]
	do_stuff<std::wstring		>("_std1.xml",		L"This is a test");
	do_stuff<std::wstring		>("_std2.xml",		L"...");

	wcout << L"---------- object::wstring (SEE _object*.xml) ----------" << endl;
	//derived as object		[OK], BUT has TAGS surrounding the string in XML
	do_stuff<object::wstring	>("_object1.xml",	L"This is a test");	
	do_stuff<object::wstring	>("_object2.xml",	L"...");

	wcout << L"---------- primitive::wstring ----------" << endl;
	//derived as primitive	[FAILS]
	do_stuff<primitive::wstring	>("_prim1.xml",		L"This is a test");
	do_stuff<primitive::wstring	>("_prim2.xml",		L"...");
	
	return 0;
}

Output:

---------- std::wstring ----------
saving: This is a test
loaded: This is a test (EQUAL)
*** Success
saving: ...
loaded: ... (EQUAL)
*** Success
---------- object::wstring (SEE _object*.xml) ----------
saving: This is a test
loaded: This is a test (EQUAL)
*** Success
saving: ...
loaded: ... (EQUAL)
*** Success
---------- primitive::wstring ----------
saving: This is a test
*** Failure: stream error***
saving: ...
loaded: ...</std_wstr> (NOT EQUAL)
*** Success

[Robert Ramey]

OK - I looked into this.

I'm not sure what you're trying to do, but here is what I've found.

The "official" std::wstring works fine. It is marked as primitive and has special code inside of basic_text_iprimitive to handle it.

Your "object:" version also works as expected. When using xml_archives tags are required for elements. There might be a way to suppress/override this but if there is I don't remember what it is.

your "primitive:" version fails with the string "This is..". Since the type is marked primitive, when it comes time to load the string the following is called:

is >> t where t is of type primitive::wstring

since primitive::wstring is derived from std::string and t is passed by reference, t get's "promoted" to std::wstring for the is >> t operation. This operation is implemented by the standard library so that it stops when it gets a space. So you don't get the whole string back. Now things are out of whack when the end tag is searched for and the program bombs.

Robert Ramey