Boost C++ Libraries: Ticket #4786: custom property_tree

attachment set

saleyn@… — Mon, 25 Oct 2010 18:22:02 GMT

attachment → boost_1_44_0.property_tree.patch

attachment set

saleyn@… — Mon, 25 Oct 2010 18:22:37 GMT

attachment → variant_property_tree.tgz

Variant property tree

attachment set

saleyn@… — Mon, 25 Oct 2010 18:56:55 GMT

attachment → boost_1_44_0.property_tree.2.patch

Please use this patch instead, the previous upload broke some legacy functionality.

viboes — Sat, 27 Nov 2010 15:51:10 GMT

Please, could you add the kind or errors you get?

Sun, 05 Dec 2010 00:34:06 GMT

You can just untar the variant_property_tree.tgz and try to compile the project with latest version of property_tree to get the errors. If that's not enough, I can post the actual errors.

anonymous — Tue, 20 Mar 2012 11:13:04 GMT

the idea looks nice. the C-style casts don't. the try/catch cascade with lexical_cast for guessing the type from string looks even more horrible, but there's nothing else to do since INFO files don't have any hints like "this property has that type" inside. it's too bad that the library maintainer didn't pay attention yet, maybe he could suggest something.

type changed

Sebastian Redl — Tue, 20 Mar 2012 15:22:45 GMT

type Bugs → Feature Requests

The INFO parser (like all other parsers) only supports string trees. That's just the way it is. I may enhance the parsers in the future. However, wildly casting pointers is *not* the way to do it.

Marking this as a feature request.

saleyn — Tue, 20 Mar 2012 15:32:33 GMT

Replying to cornedbee:

The INFO parser (like all other parsers) only supports string trees. That's just the way it is. I may enhance the parsers in the future. However, wildly casting pointers is *not* the way to do it.

Marking this as a feature request.

I agree that the casts were somewhat ugly, but supporting the feature of having derived trees with typed data nodes favored by the property_tree's underlying parsing architecture would be a great add-on. I couldn't see the way of doing it cleanly in the current implementation without needing to patch it at least minimally. So if you can introduce this feature given the derived version of the variant_tree that I submitted above it would be great!

saleyn — Tue, 20 Mar 2012 15:52:16 GMT

Replying to anonymous:

the idea looks nice. the C-style casts don't. the try/catch cascade with lexical_cast for guessing the type from string looks even more horrible, but there's nothing else to do since INFO files don't have any hints like "this property has that type" inside. it's too bad that the library maintainer didn't pay attention yet, maybe he could suggest something.

Ideally the parsers would minimally give an ability to infer the type based on the parsed data, and provide a hint to the derived implementation so that the commonly identifiable inputs "1", "2.0", '"string"' are transformed into statically overridable calls that the transform function would be able to map into properly typed objects, e.g.:

enum hint_t {UNDEFINED, INT, FLOAT, STRING};

std::pair<std::basic_string<Ch>, hint_t> raw_data = read_data(text, &need_more_lines); typename Ptree::data_type typed_data = transform_data(raw_data); last.data() = typed_data;

anonymous — Wed, 21 Mar 2012 10:07:49 GMT

that last idea of guessing the type by its looks sounds dangerous to me. it may prove even worse than the try/catch/lexical_cast cascade. it's better to provide a way to supply some "schema" to the parser so he would know each property's type. explicit is better than implicit, as they say.

Sebastian Redl — Wed, 21 Mar 2012 10:38:42 GMT

I can see the point in making the JSON parser variant-tree-aware. JSON is already typed, after all, and the current parser throws away information.

But all this talk about how to shoehorn INFO into a typed tree doesn't make sense to me. INFO is not any official format. It was invented by the original PTree author specifically for string ptrees. If you want types, then why not just adjust the format to allow type annotations?

saleyn — Wed, 21 Mar 2012 13:28:59 GMT

Replying to anonymous:

that last idea of guessing the type by its looks sounds dangerous to me. it may prove even worse than the try/catch/lexical_cast cascade. it's better to provide a way to supply some "schema" to the parser so he would know each property's type. explicit is better than implicit, as they say.

Agree. The explicit schema would be a much better solution. Especially if that schema can be uniformly adapted across all ptree's parsers.

saleyn — Wed, 21 Mar 2012 13:40:45 GMT

Replying to cornedbee:

I can see the point in making the JSON parser variant-tree-aware. JSON is already typed, after all, and the current parser throws away information.

But all this talk about how to shoehorn INFO into a typed tree doesn't make sense to me. INFO is not any official format. It was invented by the original PTree author specifically for string ptrees. If you want types, then why not just adjust the format to allow type annotations?

IMHO, INFO is a very succinct and convenient format. It's conceptually very similar to Erlang term encoding format, which is quite popular. In fact most of dynamically typed languages use standard implicit rules to parse 1234 as integers, 234.5 or 2.345e2 as floats, "1234" as strings, and true or false as booleans (just like JSON protocol does, except that it doesn't distinguish integers from floats and treats them as "number"). So, adapting that idea to the INFO parser so that it can use a custom translator to allow a user to take advantage of that extra metadata wouldn't be that bad.

saleyn — Wed, 21 Mar 2012 20:04:17 GMT

For what it's worth, I converted the variant tree example to the latest boost version 1.49.0, and it turned out that there's a way to get it working without a need for patching the source. The latest example is uploaded as attachment.

Following this thread I do like the schema idea, but this is definitely not a bug request.

attachment set

saleyn — Wed, 21 Mar 2012 20:05:23 GMT

attachment → variant_property_tree-1_49_0.tgz

Sample of a variant property tree with typed node data (boost version 1_49_0

anonymous — Fri, 13 Apr 2012 10:29:25 GMT

Some way to specify a custom default translator in basic_ptree would be really useful. I'm talking about the last example. That inheritance from basic_ptree with all those handcrafted member functions looks cumbersome (and also somewhat dangerous - there's no virtual destructor in basic_tree).

anonymous — Thu, 26 Apr 2012 07:45:22 GMT

a few more notes regarding your example. util::detail::variant_translator::get_value seems to lack support for converting the internal value to std::string (except for the case when its type is also std::string). and if you pass std::string to util::detail::variant_translator::put_value, it looks like it will just replace the internal value with the supplied string, changing its type to std::string, not converting the string to the current internal type. maybe it's better to add util::detail::variant_translator<std::string> specialization to improve this. correct me if I've missed anything. :)

attachment set

saleyn — Thu, 26 Apr 2012 10:39:28 GMT

attachment → variant_tree_boost_1_49_0.tgz

Version working without needing to patch boost

saleyn — Thu, 26 Apr 2012 10:52:14 GMT

Upon checking my last submission, I noticed that I uploaded a stale version which still didn't work without patching the tree. variant_tree_boost_1_49_0.tgz solves that problem. The work-around was to apply a "post-conversion" transform after reading data from the info file to convert strings to corresponding variant types.

Concerning the last comment, the variant_translator has the following code that makes sure that std::string is one of the types it handles (variant::valid_types type includes std::string):

        template<typename T>
        typename boost::disable_if<
            boost::is_same<
                boost::mpl::end<valid_types>::type,
                boost::mpl::find<variant::valid_types, T>
            >,
            internal_type>::type
        put_value(T value) const {
            return variant(value);
        }

Since variant_tree::put_value(value) uses default translator, it should be able to handle std::string conversion. If you don't believe it does, please include a test case in the test_variant.cpp that illustrates a failing case.

anonymous — Fri, 27 Apr 2012 06:23:37 GMT

hmm, well, I see a two-argument variant constructor that calls from_string (this seems to be the only place where the conversion from string takes place), but I can't figure out where is that constructor used :)