Boost C++ Libraries: Ticket #8512: set member_hook access violation

attachment set

toby_toby_toby@… — Sun, 28 Apr 2013 20:04:07 GMT

attachment → Source1.cpp

sorry for unformatted source - readded as separate file

toby_toby_toby@… — Sun, 28 Apr 2013 20:46:42 GMT

also must add that standard offsetof macro works well (it's based on getting address of member variable of NULL-object)

Ion Gaztañaga — Sun, 28 Apr 2013 22:36:47 GMT

Thanks for the report. MSVC pointer to member handling it's quite chaotic. Can you try the following patch?

diff U3 C:/Data/Libs/LocalSVN/boost-trunk/boost/intrusive/detail/parent_from_member.hpp C:/Data/Libs/boost/boost/intrusive/detail/parent_from_member.hpp
--- C:/Data/Libs/LocalSVN/boost-trunk/boost/intrusive/detail/parent_from_member.hpp     Sat Nov 24 09:48:40 2012
+++ C:/Data/Libs/boost/boost/intrusive/detail/parent_from_member.hpp    Sun Apr 28 23:53:19 2013
@@ -37,7 +37,15 @@
       boost::int32_t offset;
    } caster;
    caster.ptr_to_member = ptr_to_member;
-   return std::ptrdiff_t(caster.offset);
+   //MSVC ABI can use up to 3 int32 to represent pointer to member data
+   //with virtual base classes, in that case we must
+   //add the size of the extra __vfptr vtable.
+   if(sizeof(caster) > sizeof(boost::int32_t)){
+      return std::ptrdiff_t(caster.offset) + sizeof(void*);
+   }
+   else{
+      return std::ptrdiff_t(caster.offset);
+   }
    //This works with gcc, msvc, ac++, ibmcpp
    #elif defined(__GNUC__)   || defined(__HP_aCC) || defined(BOOST_INTEL) || \
          defined(__IBMCPP__) || defined(__DECCXX)

toby_toby_toby@… — Mon, 29 Apr 2013 00:23:26 GMT

Thanks, it helped.

PS: But I've got a feeling that previously I was able to make 8-byte difference (by playing with inheritance/member vars/additional virtual functions). I didn't payed attention because I thought that any difference is the same bad thing, that fix will not be for 4-byte only. Now I'm trying to simulate this 8-byte difference again. I will confirm any result, even if I was mistaken and there were only 4-byte errors.

anonymous — Mon, 29 Apr 2013 00:55:09 GMT

Well, now it's broken in different place (intrusive::list_member_hook).

We hit patched condition "if(sizeof(caster) > sizeof(boost::int32_t))" and add sizeof(void*), 8+4=12, but actual difference is 8.

Now I'm gonna update source to include check for this also.

PS: Sorry if this question is dumb, but why don't we use VC/stddef.h offsetof or it's approach with getting address of member of null-object?

attachment set

toby_toby_toby@… — Mon, 29 Apr 2013 01:06:47 GMT

attachment → set_and_list.cpp

after patch set is working, but list is not

toby_toby_toby@… — Mon, 29 Apr 2013 11:43:44 GMT

Switched to base hooks with tags, because fixing of member hook failed:

1) offsetof's approach not working - access violation, null can't be dereferenced

2) fake object "char temp[sizeof(Parent)]" also not working - it actually accesses vptr

3) inspection of additional integers in caster gave nothing - they are the same for both cases, no way to deduce

Ion Gaztañaga — Mon, 29 Apr 2013 15:45:54 GMT

I don't know if virtual or multiple inheritance is involved in these issues, and it seems that MSVC's pointer to member data can have the size of 1, 2 or 3 ints.

Can you please provide a test case with a 8 byte difference? I guess it could help me to deduce MSVC's ABI.

toby_toby_toby@… — Mon, 29 Apr 2013 18:56:52 GMT

I was unable to simulate 8 byte difference, it may never exist in fact. I just had feeling that it was, but it seems that I was wrong.

But set_and_list.cpp attachment gives you an example when your current patch is not working, so you can try to deduce something on this.. I was unable to. set_item is working fine with +sizeof(void*), but list_item gives wrong pointer with this void* addon.

Ion Gaztañaga — Tue, 30 Apr 2013 22:48:15 GMT

It seems that even there is no virtual inheritance boundary between the member and the parent, MSVC ABI requires pointer adjustment operations with information from the virtual base table. asm code for pointer to member dereferencing shows vbtable is read to correct parent pointer before adding the offset stored in the pointer to member data.

So it seems very difficult or maybe impossible, to obtain the pointer adjustment value from the vtable to go back from the member to the parent (we need a way to obtain access to the vbtable without using a valid parent pointer).

I think the library should give a compile time error (if pointer to member is not a simple offset) and document that for Windows compilers member hooks have limitations and don't work if types have direct or indirect virtual bases.

toby_toby_toby@… — Tue, 30 Apr 2013 23:25:36 GMT

Static assertion is a good idea, even better with unit tests.

status changed; resolution set

Ion Gaztañaga — Tue, 28 May 2013 21:01:25 GMT

status new → closed
resolution → wontfix

After some research even reverse engineering MSVC RTTI we might not solve this issue. Boost 1.54 will include the static assertion for member hooks that need complex pointers to member data structures in MSVC ABI. Thanks for the report.

toby_toby_toby@… — Sat, 08 Jun 2013 08:31:17 GMT

Just an idea, add new option to hook:

typedef boost::intrusive::list_member_hook<
  boost::intrusive::store_pointer_to_parent_inside<true>
  >

So we will have

parent* _parent;

inside hook (null in constructor) for platforms that require it, if GCC does not need - then just do nothing with this option.

Then, when we call container.insert( item ), we actually have reference to item so we can assign it:

(item.*hook_member_ptr)._parent = &item;

And clear on erase. Now we don't need offsetoff or etc.

Static assertion will only be rised when compiler requires this (for people like me who actually expirienced problems). For other users it will be backward compatible: no need to specify option, sizeof same. Option only required for msvc+virtual inheritance.