Boost C++ Libraries: Ticket #5876: Serialization - tracking of non-versioned classes

attachment set

ybungalobill@… — Thu, 08 Sep 2011 09:44:45 GMT

attachment → ATest.cpp

code demonstrating the problem

ybungalobill@… — Thu, 15 Sep 2011 15:55:55 GMT

Hereby, I submit a patch that fixes the above bug. However, it doesn't solve the issue completely since it still suffers from the problem that if someone serializes an object with implementation level of class-info through a pointer, then all other instances are are tracked too (even those that are not serialized through a pointer). However it's now trivial to fix the behavior to be the intended one, it's just that it involves archive format breakage so we must bump the library version.

attachment set

ybungalobill@… — Thu, 15 Sep 2011 15:57:43 GMT

attachment → boost.serrialization.patch

proposed fix

attachment set

ybungalobill@… — Sun, 18 Sep 2011 07:54:11 GMT

attachment → boost.serialization2.patch

I apologize, the previous fix doesn't fix the save correctly.

Robert Ramey — Mon, 23 Jan 2012 16:56:11 GMT

Looks like you're on to something here. I'm still looking at this.

"suffers from the problem that if someone serializes an object with implementation level of class-info through a pointer, then all other instances are are tracked too (even those that are not serialized through a pointer)"

But this is the intended behavior! If this wasn't true, pointers to objects previously serialized as objects wouldn't be restored properly.

Robert Ramey

ybungalobill@… — Mon, 23 Jan 2012 17:45:16 GMT

But this is the intended behavior! If this wasn't true, pointers to objects previously serialized as objects wouldn't be restored properly.

I find it counter-intuitive. I expect objects serialized as objects to mean 'by value' and not be tracked, while objects serialized by pointer to mean 'by reference' and be tracked. Anyway it's your choice and I won't criticize it.

The problem is that you can't implement the intended behavior properly for object_serializable. The above code is a minimal code that reproduces the problem, but it's not hypothetical. The real world scenario went like this:

3rd party interprocess communication library used serialization by pointer in generic code. It happened that we passed vector<int> between two processes, so it serialized it through vector<int>*. Another unrelated part of our codebase serialized shared_ptr<vector<int>> which in turn is serialized through vector<int>*. Note that nowhere the user (me) played with the implementation level. Each part is valid in isolation. So far so good.

We updated from boost 1.33.1 to boost 1.47.0. It got broken. Why? I don't exactly know. My hypothesis is that because the way singletons are created had changed (function statics in 1.33.1 versus class statics in 1.47.0) the linker handles them differently.

Whatever the reason I think that code that breaks depending on the existence of some unrelated other code is worse than the inability to serialize object_serializable object as object and then track it through a pointer.

Thank you,
Yakov Galka

cc set

Fri, 29 Jun 2012 13:58:07 GMT

cc mika.fischer@… added

status changed

Robert Ramey — Sun, 15 Jul 2012 20:44:02 GMT

status new → assigned

"I find it counter-intuitive. I expect objects serialized as objects to mean 'by value' and not be tracked, while objects serialized by pointer to mean 'by reference' and be tracked. Anyway it's your choice and I won't criticize it."

This was a design decision made at the very beginning as an optimization. Basically the idea was that there was no need to include the pointer deserialization code unless the the item was serialized explicitly via a pointer. In retrospect, this turns out to be questionable. It's possible for the same object to be serialized multiple times even though it's never through a pointer. Actually, this needs to be re-thought in detail - which I'm not likely to do any time soon.

One thing you might consider is explicitly setting the implementation level to include tracking so that items get tracked regardless of how they are used.

Robert Ramey

ybungalobill@… — Tue, 05 Apr 2016 10:51:57 GMT

Hey, it has been a long time ago, yet I found a draft somewhere in my inbox:

Changing the implementation level of 'vector<int>' as you suggest would break backward and forward compatibility, due to the exact same problem. Also it counts as boost modification, and if we agree to resort to changing boost, then there are better alternatives: fix the core problem. We had already applied a bunch of mandatory modifications (a fix for #5499 and full support for saving boost 1.33.1 archive format), so using the patch I sent is the most practical choice. There are still some circumstances that may trigger the bug (that I can't recall now), but thankfully it isn't the case in our code. We managed to read and write archives binary identical to those of boost 1.33.1.

I'm sorry to say that, but since then we stopped using Boost.Serialization in new code and hope to obliterate it in other places one day, which is tricky due to the deployed systems life cycle.

Yakov Galka