Boost C++ Libraries: Ticket #11204: undefined behavior sanitizer complains about runtime_error thrown in serialization/singleton.hpp before main()

summary changed

Sat, 18 Apr 2015 20:33:07 GMT

summary undefined behavior sanitizer complains about serialization/singleton.hpp thrown before main() → undefined behavior sanitizer complains about runtime_error thrown in serialization/singleton.hpp before main()

status changed; resolution set

Robert Ramey — Sun, 24 May 2015 00:06:51 GMT

status new → closed
resolution → invalid

I wasn't able to get the (address?) sanitizer working in my Xcode 6 environment. But I did trace through the program with the debugger. line 132 of singleton.hpp contains the line

use(instance);

where the value of instance has the (initialized value of zero). This might be considered an error by the sanitizer, but use(...) is an empty function. It is only included to guarantee that the singleton is called before main. If this isn't included, compile for release may optimize away the pre-main invocation and the class may not function as intended. So I would call this an over zealous behavior of the sanitizer. Or maybe the sanitizer could be considered not smart enough to trace one more level deep. or... what ever.

Robert Ramey — Sun, 24 May 2015 16:46:58 GMT

News flash - I think I found and easy fix for this. Right now it's only on my own machine, but I expect to be uploading to Develop branch in the near future.

tm@… — Wed, 02 Dec 2015 19:55:08 GMT

Replying to ramey:

News flash - I think I found and easy fix for this. Right now it's only on my own machine, but I expect to be uploading to Develop branch in the near future.

Just a short nudge on this issue as i've tried to build our project with UBSAN and hit the same issue. Do you still have the easy fix and could you upload it?

anonymous — Wed, 13 Jan 2016 15:21:07 GMT

I think what he means is this:

https://github.com/boostorg/serialization/commit/4f83fc163afad8fc078afcb0eccfbda1f161b3b6

 private:
     BOOST_DLLEXPORT static T & instance;
     // include this to provoke instantiation at pre-execution time
-    static void use(T const &) {}
+    static void use(T const *) {}
     BOOST_DLLEXPORT static T & get_instance() {
         static detail::singleton_wrapper< T > t;
         // refer to instance, causing it to be instantiated (and
         // initialized at startup on working compilers)
         BOOST_ASSERT(! detail::singleton_wrapper< T >::m_is_destroyed);
-        use(instance);
+        use(& instance);
         return static_cast<T &>(t);
     }
 public:

Unfortunately, it didn't fix the problem for me, or rather fixed it, but only partially. Now I'm getting another error, along the lines of:

==29772==ERROR: AddressSanitizer: SEGV on unknown address 0x000000000008 (pc 0x7fca1b12c4d0 bp 0x7ffcb629d230 sp 0x7ffcb629d200 T0)
    #0 0x7fca1b12c4cf in boost_1_57_0::serialization::void_cast_detail::void_caster_primitive<tt::InstanceValueKey, tt::InstanceValue>::void_caster_primitive() (/build/debug/cpp/libotdscpp.so+0x4acd4cf)
    #1 0x7fca1b1275e6 in boost_1_57_0::serialization::detail::singleton_wrapper<boost_1_57_0::serialization::void_cast_detail::void_caster_primitive<tt::InstanceValueKey, tt::InstanceValue> >::singleton_wrapper() lib/boost/install/include/boost/serialization/singleton.hpp:106
    #2 0x7fca1b1276f1 in boost_1_57_0::serialization::singleton<boost_1_57_0::serialization::void_cast_detail::void_caster_primitive<tt::InstanceValueKey, tt::InstanceValue> >::get_instance() lib/boost/install/include/boost/serialization/singleton.hpp:128
    #3 0x7fca1b1124b4 in __static_initialization_and_destruction_0 lib/boost/install/include/boost/serialization/singleton.hpp:149
    #4 0x7fca1b1141c5 in _GLOBAL__sub_I_instancevalue.cc cpp/instancevalue.cc:659
    #5 0x7fca28106139  (/lib64/ld-linux-x86-64.so.2+0x10139)
    #6 0x7fca28106222  (/lib64/ld-linux-x86-64.so.2+0x10222)
    #7 0x7fca2810ac6f  (/lib64/ld-linux-x86-64.so.2+0x14c6f)
    #8 0x7fca28105ff3  (/lib64/ld-linux-x86-64.so.2+0xfff3)
    #9 0x7fca2810a3ba  (/lib64/ld-linux-x86-64.so.2+0x143ba)
    #10 0x7fca26d6a02a  (/lib/x86_64-linux-gnu/libdl.so.2+0x102a)
    #11 0x7fca28105ff3  (/lib64/ld-linux-x86-64.so.2+0xfff3)
    #12 0x7fca26d6a62c  (/lib/x86_64-linux-gnu/libdl.so.2+0x162c)
    #13 0x7fca26d6a0c0 in dlopen (/lib/x86_64-linux-gnu/libdl.so.2+0x10c0)
    #14 0x7fca271c23ef in dlopen (/usr/lib/gcc/x86_64-linux-gnu/5/libasan.so+0x373ef)
...

anonymous — Fri, 15 Jan 2016 15:02:14 GMT

Apparently, the best way to solve this would be simply to disable instrumentation for this case:

#if defined(__clang__) || defined (__GNUC__)
# define ATTRIBUTE_NO_SANITIZE_ADDRESS __attribute__((no_sanitize_address))
#else
# define ATTRIBUTE_NO_SANITIZE_ADDRESS
#endif
...
ATTRIBUTE_NO_SANITIZE_ADDRESS
void ThisFunctionWillNotBeInstrumented() {...}

Unfortunately, I wasn't able to find the combination of methods I need to mark so far to make it work :-/

status changed; resolution deleted

Robert Ramey — Fri, 15 Jan 2016 17:44:11 GMT

status closed → reopened
resolution invalid

anonymous — Wed, 27 Jan 2016 14:06:26 GMT

Okay, an update on this ticket: I have finally managed to have another look at it, and here are the conclusions:

1) I was wrong to say that ramey's fix (see comment:5) does not resolve the original problem of the reporter (as in the UBSan still complains).

It does work indeed for me now on GCC 5.3 with UBSan enabled, it's just that after fixing the reported problem, the program would still crash if ASan is also enabled due to a different problem.

2) The source of the second problem is in void_caster and it is actually obvious if one attentively reads the backtrace from ASan and the code.

ASan doesn't like accesses to unknown addresses: AddressSanitizer: SEGV on unknown address 0x000000000008. This is caused by the constructor called just before the use() function and the following patch makes the problem go away:

--- boost/serialization/void_cast.hpp        2016-01-27 14:46:23.000000000 +0100
+++ boost/serialization/void_cast.hpp.orig   2016-01-18 11:50:19.000000000 +0100
@@ -180,7 +180,14 @@
 void_caster_primitive<Derived, Base>::void_caster_primitive() :
     void_caster(
         & type_info_implementation<Derived>::type::get_const_instance(),
-        & type_info_implementation<Base>::type::get_const_instance()
+        & type_info_implementation<Base>::type::get_const_instance(),
+        // note:I wanted to displace from 0 here, but at least one compiler
+        // treated 0 by not shifting it at all.
+        reinterpret_cast<std::ptrdiff_t>(
+            static_cast<Derived *>(
+                reinterpret_cast<Base *>(8)
+            )
+        ) - 8
     )
 {
     recursive_register();

After seeing this code, I thinks it's quite obvious where the magical 0x000000000008 comes from...

So, having this said, the original problem is indeed solved, and an additional problem is at least diagnosed. Shall we keep this ticket open for this additional problem, or you would rather like to close it and have me open a new ticket for this second issue?

Of course, the workaround above works for me right now, but it would be great to have a better solution out of the box.

Thanks for your work on boost::serialization!

anonymous — Mon, 22 Aug 2016 11:48:43 GMT

I've just checked with clang++-3.9 nightly + ASan, and, unfortunately, void_caster_primitive still causes a SEGV. Is there anything that can be done about it? The comment doesn't specify which compiler specifically was the problematic one and so I'm not sure whether just removing the code for GCC / Clang is "safe".

davidlt — Wed, 13 Dec 2017 14:57:26 GMT

I am using boost 1.63.0, GCC 7.2.1, bintutils 2.29.1 and trying to compile software using boost with Undefined Behavior Sanitizer (UBSan). I think, am hitting the same issue.

Compile generates: mov (%r12),%rax and registers are:

rax            0x8      8
r12            0x8      8

Here is a full picture:

   ┌──/cvmfs/cms-ib.cern.ch/nweek-02500/slc6_amd64_gcc700/external/boost/1.63.0-mmelna2/include/boost/serialization/void_cast.hpp──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┐
   │184             // note:I wanted to displace from 0 here, but at least one compiler                                                                                                                                                                                        │
   │185             // treated 0 by not shifting it at all.                                                                                                                                                                                                                    │
   │186             reinterpret_cast<std::ptrdiff_t>(                                                                                                                                                                                                                          │
  >│187                 static_cast<Derived *>(                                                                                                                                                                                                                                │
   │188                     reinterpret_cast<Base *>(8)                                                                                                                                                                                                                        │
   │189                 )                                                                                                                                                                                                                                                      │
   │190             ) - 8                                                                                                                                                                                                                                                      │
   │191         )                                                                                                                                                                                                                                                              │
   │192     {                                                                                                                                                                                                                                                                  │
   │193         recursive_register();                                                                                                                                                                                                                                          │
   │194     }                                                                                                                                                                                                                                                                  │
   │195                                                                                                                                                                                                                                                                        │
   │196     template <class Derived, class Base>                                                                                                                                                                                                                               │
   │197     void_caster_primitive<Derived, Base>::~void_caster_primitive(){                                                                                                                                                                                                    │
   │198         recursive_unregister();                                                                                                                                                                                                                                        │
   │199     }                                                                                                                                                                                                                                                                  │
   │200                                                                                                                                                                                                                                                                        │
   │201     template <class Derived, class Base>                                                                                                                                                                                                                               │
   │202     class BOOST_SYMBOL_VISIBLE void_caster_virtual_base :                                                                                                                                                                                                              │
   │203         public void_caster                                                                                                                                                                                                                                             │
   │204     {                                                                                                                                                                                                                                                                  │
   ┌───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┐
   │0x7fffe6c4461c <boost::serialization::void_cast_detail::void_caster_primitive<cond::GenericSummary, cond::Summary>::void_caster_primitive()+138>        mov    $0x8,%r12d                                                                                                  │
   │0x7fffe6c44622 <boost::serialization::void_cast_detail::void_caster_primitive<cond::GenericSummary, cond::Summary>::void_caster_primitive()+144>        mov    %r12,%rax                                                                                                   │
   │0x7fffe6c44625 <boost::serialization::void_cast_detail::void_caster_primitive<cond::GenericSummary, cond::Summary>::void_caster_primitive()+147>        test   %r12,%r12                                                                                                   │
   │0x7fffe6c44628 <boost::serialization::void_cast_detail::void_caster_primitive<cond::GenericSummary, cond::Summary>::void_caster_primitive()+150>        je     0x7fffe6c44635 <boost::serialization::void_cast_detail::void_caster_primitive<cond::GenericSummary, cond::Su│
   │0x7fffe6c4462a <boost::serialization::void_cast_detail::void_caster_primitive<cond::GenericSummary, cond::Summary>::void_caster_primitive()+152>        mov    %rax,%rdx                                                                                                   │
   │0x7fffe6c4462d <boost::serialization::void_cast_detail::void_caster_primitive<cond::GenericSummary, cond::Summary>::void_caster_primitive()+155>        and    $0x7,%edx                                                                                                   │
   │0x7fffe6c44630 <boost::serialization::void_cast_detail::void_caster_primitive<cond::GenericSummary, cond::Summary>::void_caster_primitive()+158>        test   %rdx,%rdx                                                                                                   │
   │0x7fffe6c44633 <boost::serialization::void_cast_detail::void_caster_primitive<cond::GenericSummary, cond::Summary>::void_caster_primitive()+161>        je     0x7fffe6c44644 <boost::serialization::void_cast_detail::void_caster_primitive<cond::GenericSummary, cond::Su│
   │0x7fffe6c44635 <boost::serialization::void_cast_detail::void_caster_primitive<cond::GenericSummary, cond::Summary>::void_caster_primitive()+163>        mov    %rax,%rsi                                                                                                   │
   │0x7fffe6c44638 <boost::serialization::void_cast_detail::void_caster_primitive<cond::GenericSummary, cond::Summary>::void_caster_primitive()+166>        lea    0x15f221(%rip),%rdi        # 0x7fffe6da3860                                                                 │
   │0x7fffe6c4463f <boost::serialization::void_cast_detail::void_caster_primitive<cond::GenericSummary, cond::Summary>::void_caster_primitive()+173>        callq  0x7fffe6bd73a0 <__ubsan_handle_type_mismatch@plt>                                                           │
  >│0x7fffe6c44644 <boost::serialization::void_cast_detail::void_caster_primitive<cond::GenericSummary, cond::Summary>::void_caster_primitive()+178>        mov    (%r12),%rax                                                                                                 │
   │0x7fffe6c44648 <boost::serialization::void_cast_detail::void_caster_primitive<cond::GenericSummary, cond::Summary>::void_caster_primitive()+182>        mov    %rax,%rdx                                                                                                   │
   │0x7fffe6c4464b <boost::serialization::void_cast_detail::void_caster_primitive<cond::GenericSummary, cond::Summary>::void_caster_primitive()+185>        movabs $0xd36d9746bc2d02eb,%rax                                                                                    │
   │0x7fffe6c44655 <boost::serialization::void_cast_detail::void_caster_primitive<cond::GenericSummary, cond::Summary>::void_caster_primitive()+195>        xor    %rdx,%rax                                                                                                   │
   │0x7fffe6c44658 <boost::serialization::void_cast_detail::void_caster_primitive<cond::GenericSummary, cond::Summary>::void_caster_primitive()+198>        mov    %rax,%rcx                                                                                                   │
   │0x7fffe6c4465b <boost::serialization::void_cast_detail::void_caster_primitive<cond::GenericSummary, cond::Summary>::void_caster_primitive()+201>        movabs $0x9ddfea08eb382d69,%rax                                                                                    │
   │0x7fffe6c44665 <boost::serialization::void_cast_detail::void_caster_primitive<cond::GenericSummary, cond::Summary>::void_caster_primitive()+211>        imul   %rcx,%rax                                                                                                   │
   │0x7fffe6c44669 <boost::serialization::void_cast_detail::void_caster_primitive<cond::GenericSummary, cond::Summary>::void_caster_primitive()+215>        mov    %rax,%rcx                                                                                                   │
   │0x7fffe6c4466c <boost::serialization::void_cast_detail::void_caster_primitive<cond::GenericSummary, cond::Summary>::void_caster_primitive()+218>        shl    $0x2f,%rcx                                                                                                  │
   │0x7fffe6c44670 <boost::serialization::void_cast_detail::void_caster_primitive<cond::GenericSummary, cond::Summary>::void_caster_primitive()+222>        xor    %rdx,%rax                                                                                                   │
   └───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┘
multi-thre Thread 0x7ffff7dc97 In: boost::serialization::void_cast_detail::void_caster_primitive<cond::GenericSummary, cond::Summary>::void_caster_primitive                                                                                           L187  PC: 0x7fffe6c44644
Starting program: /cvmfs/cms-ib.cern.ch/nweek-02500/slc6_amd64_gcc700/cms/cmssw/CMSSW_10_0_UBSAN_X_2017-11-27-2300/external/slc6_amd64_gcc700/bin/python /cvmfs/cms-ib.cern.ch/week0/slc6_amd64_gcc700/cms/cmssw/CMSSW_10_0_UBSAN_X_2017-11-27-2300/src/FWCore/Utilities/scripts
/edmCheckClassVersion -l tmp/slc6_amd64_gcc700/src/CondFormats/Common/src/CondFormatsCommon/libCondFormatsCommon.so -x src/CondFormats/Common/src/classes_def.xml
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib64/libthread_db.so.1".
warning: File "/cvmfs/cms-ib.cern.ch/nweek-02500/slc6_amd64_gcc700/external/gcc/7.0.0-mmelna2/lib64/libstdc++.so.6.0.24-gdb.py" auto-loading has been declined by your `auto-load safe-path' set to "$debugdir:$datadir/auto-load".
Program received signal SIGSEGV, Segmentation fault.
0x00007fffe6c44644 in boost::serialization::void_cast_detail::void_caster_primitive<cond::GenericSummary, cond::Summary>::void_caster_primitive (
    this=0x7fffe6e18a40 <boost::serialization::singleton<boost::serialization::void_cast_detail::void_caster_primitive<cond::GenericSummary, cond::Summary> >::get_instance()::t>)
    at /cvmfs/cms-ib.cern.ch/nweek-02500/slc6_amd64_gcc700/external/boost/1.63.0-mmelna2/include/boost/serialization/void_cast.hpp:187
(gdb) p $_siginfo._sifields._sigfault.si_addr
$1 = (void *) 0x8
(gdb)
rax            0x8      8
rbx            0x7fffe6e18a40   140737066928704
rcx            0x14     20
rdx            0x0      0
rsi            0x12     18
rdi            0x7fffe6e18a40   140737066928704
rbp            0x7fffffff2660   0x7fffffff2660
rsp            0x7fffffff2640   0x7fffffff2640
r8             0x24ebc30        38714416
r9             0x6      6
r10            0x7fffffff23f0   140737488298992
r11            0x7fffe6c44592   140737065010578
r12            0x8      8
r13            0x7fffe6d02a98   140737065790104
r14            0x7fffffff3a20   140737488304672
r15            0x0      0
rip            0x7fffe6c44644   0x7fffe6c44644 <boost::serialization::void_cast_detail::void_caster_primitive<cond::GenericSummary, cond::Summary>::void_caster_primitive()+178>
eflags         0x10246  [ PF ZF IF RF ]
cs             0x33     51
ss             0x2b     43
ds             0x0      0
es             0x0      0

matteo@… — Sat, 20 Jan 2018 08:49:48 GMT

I must admit that after trying to fiddle with that function quite a bit, I resorted to just adding __attribute__ ((no_sanitize_undefined)) to void_caster_primitive<Derived, Base>::void_caster_primitive(). That shows at least that this in still a current problem.