Opened 8 years ago
Last modified 5 years ago
#11204 reopened Bugs
undefined behavior sanitizer complains about runtime_error thrown in serialization/singleton.hpp before main()
| Reported by: | Owned by: | Robert Ramey | |
|---|---|---|---|
| Milestone: | To Be Determined | Component: | serialization |
| Version: | Boost Development Trunk | Severity: | Problem |
| Keywords: | Cc: |
Description
How to reproduce:
#include <boost/archive/text_iarchive.hpp>
#include <boost/archive/text_oarchive.hpp>
#include <boost/serialization/vector.hpp>
#include <vector>
#include <fstream>
using namespace std;
struct Data {
vector<int> v;
};
namespace boost {
namespace serialization {
template<class Archive>
void serialize(Archive & a, Data &d, const unsigned int version)
{
a & d.v;
}
}
}
int main(int argc, char **argv)
{
if (argc > 10) {
ifstream f("/dev/null");
boost::archive::text_iarchive a(f);
Data d;
a >> d;
} else {
ofstream f("/dev/null");
boost::archive::text_oarchive a(f);
Data d;
a << d;
}
return 0;
}
Compile via:
$ g++ -g -std=c++11 -I/home/juser/src/boost/modular-boost \ -L/home/juser/src/boost/modular-boost/stage/lib \ -Wl,-R/home/juser/src/boost/modular-boost/stage/lib \ -fsanitize=undefined test_serialize.cc \ -o test_serialize -lboost_serialization
(GCC's undefined behavior sanitizer is enabled with -fsanitize=undefined)
Run:
$ ./test_serialize
Expected output:
(nothing)
Actual output:
/home/juser/src/boost/modular-boost/boost/serialization/singleton.hpp:132:21:
runtime error: reference binding to null pointer of type
'const struct extended_type_info_typeid'
/home/juser/src/boost/modular-boost/boost/serialization/singleton.hpp:132:21:
runtime error: reference binding to null pointer of type
'const struct iserializer'
/home/juser/src/boost/modular-boost/boost/serialization/singleton.hpp:132:21: runtime error: reference binding to null pointer of type 'const struct oserializer'
/home/juser/src/boost/modular-boost/boost/serialization/singleton.hpp:132:21: runtime error: reference binding to null pointer of type 'const struct extended_type_info_typeid'
/home/juser/src/boost/modular-boost/boost/serialization/singleton.hpp:132:21: runtime error: reference binding to null pointer of type 'const struct oserializer'
/home/juser/src/boost/modular-boost/boost/serialization/singleton.hpp:132:21: runtime error: reference binding to null pointer of type 'const struct iserializer'
First backtrace when breaking in singleton.hpp:132:
(gdb) bt
#0 boost::serialization::singleton<boost::serialization::extended_type_info_typeid<Data> >::get_instance ()
at /home/juser/src/boost/modular-boost/boost/serialization/singleton.hpp:132
#1 0x0000000000407ebd in boost::serialization::singleton<boost::serialization::extended_type_info_typeid<Data> >::get_const_instance ()
at /home/juser/src/boost/modular-boost/boost/serialization/singleton.hpp:141
#2 0x0000000000407924 in boost::archive::detail::iserializer<boost::archive::text_iarchive, Data>::iserializer (
this=0x640a60 <boost::serialization::singleton<boost::archive::detail::iserializer<boost::archive::text_iarchive, Data> >::get_instance()::t>)
at /home/juser/src/boost/modular-boost/boost/archive/detail/iserializer.hpp:128
#3 0x0000000000407373 in boost::serialization::detail::singleton_wrapper<boost::archive::detail::iserializer<boost::archive::text_iarchive, Data> >::singleton_wrapper (
this=0x640a60 <boost::serialization::singleton<boost::archive::detail::iserializer<boost::archive::text_iarchive, Data> >::get_instance()::t>)
at /home/juser/src/boost/modular-boost/boost/serialization/singleton.hpp:106
#4 0x000000000040740b in boost::serialization::singleton<boost::archive::detail::iserializer<boost::archive::text_iarchive, Data> >::get_instance ()
at /home/juser/src/boost/modular-boost/boost/serialization/singleton.hpp:128
#5 0x0000000000404a13 in __static_initialization_and_destruction_0 (__initialize_p=1, __priority=65535)
at /home/juser/src/boost/modular-boost/boost/serialization/singleton.hpp:149
#6 0x0000000000404c6e in _GLOBAL__sub_I_main () at test_serialize.cc:43
#7 0x000000000041abbd in __libc_csu_init ()
#8 0x00007ffff62e5f6f in __libc_start_main (main=0x4047c6 <main(int, char**)>, argc=1, argv=0x7fffffffdfb8, init=0x41ab70 <__libc_csu_init>, fini=<optimized out>,
rtld_fini=<optimized out>, stack_end=0x7fffffffdfa8) at libc-start.c:245
#9 0x00000000004046f9 in _start ()
Change History (11)
comment:1 by , 8 years ago
| Summary: | undefined behavior sanitizer complains about serialization/singleton.hpp thrown before main() → undefined behavior sanitizer complains about runtime_error thrown in serialization/singleton.hpp before main() |
|---|
comment:2 by , 7 years ago
| Resolution: | → invalid |
|---|---|
| Status: | new → closed |
follow-up: 4 comment:3 by , 7 years ago
News flash - I think I found and easy fix for this. Right now it's only on my own machine, but I expect to be uploading to Develop branch in the near future.
comment:4 by , 7 years ago
Replying to ramey:
News flash - I think I found and easy fix for this. Right now it's only on my own machine, but I expect to be uploading to Develop branch in the near future.
Just a short nudge on this issue as i've tried to build our project with UBSAN and hit the same issue. Do you still have the easy fix and could you upload it?
comment:5 by , 7 years ago
I think what he means is this:
https://github.com/boostorg/serialization/commit/4f83fc163afad8fc078afcb0eccfbda1f161b3b6
private:
BOOST_DLLEXPORT static T & instance;
// include this to provoke instantiation at pre-execution time
- static void use(T const &) {}
+ static void use(T const *) {}
BOOST_DLLEXPORT static T & get_instance() {
static detail::singleton_wrapper< T > t;
// refer to instance, causing it to be instantiated (and
// initialized at startup on working compilers)
BOOST_ASSERT(! detail::singleton_wrapper< T >::m_is_destroyed);
- use(instance);
+ use(& instance);
return static_cast<T &>(t);
}
public:
Unfortunately, it didn't fix the problem for me, or rather fixed it, but only partially. Now I'm getting another error, along the lines of:
==29772==ERROR: AddressSanitizer: SEGV on unknown address 0x000000000008 (pc 0x7fca1b12c4d0 bp 0x7ffcb629d230 sp 0x7ffcb629d200 T0)
#0 0x7fca1b12c4cf in boost_1_57_0::serialization::void_cast_detail::void_caster_primitive<tt::InstanceValueKey, tt::InstanceValue>::void_caster_primitive() (/build/debug/cpp/libotdscpp.so+0x4acd4cf)
#1 0x7fca1b1275e6 in boost_1_57_0::serialization::detail::singleton_wrapper<boost_1_57_0::serialization::void_cast_detail::void_caster_primitive<tt::InstanceValueKey, tt::InstanceValue> >::singleton_wrapper() lib/boost/install/include/boost/serialization/singleton.hpp:106
#2 0x7fca1b1276f1 in boost_1_57_0::serialization::singleton<boost_1_57_0::serialization::void_cast_detail::void_caster_primitive<tt::InstanceValueKey, tt::InstanceValue> >::get_instance() lib/boost/install/include/boost/serialization/singleton.hpp:128
#3 0x7fca1b1124b4 in __static_initialization_and_destruction_0 lib/boost/install/include/boost/serialization/singleton.hpp:149
#4 0x7fca1b1141c5 in _GLOBAL__sub_I_instancevalue.cc cpp/instancevalue.cc:659
#5 0x7fca28106139 (/lib64/ld-linux-x86-64.so.2+0x10139)
#6 0x7fca28106222 (/lib64/ld-linux-x86-64.so.2+0x10222)
#7 0x7fca2810ac6f (/lib64/ld-linux-x86-64.so.2+0x14c6f)
#8 0x7fca28105ff3 (/lib64/ld-linux-x86-64.so.2+0xfff3)
#9 0x7fca2810a3ba (/lib64/ld-linux-x86-64.so.2+0x143ba)
#10 0x7fca26d6a02a (/lib/x86_64-linux-gnu/libdl.so.2+0x102a)
#11 0x7fca28105ff3 (/lib64/ld-linux-x86-64.so.2+0xfff3)
#12 0x7fca26d6a62c (/lib/x86_64-linux-gnu/libdl.so.2+0x162c)
#13 0x7fca26d6a0c0 in dlopen (/lib/x86_64-linux-gnu/libdl.so.2+0x10c0)
#14 0x7fca271c23ef in dlopen (/usr/lib/gcc/x86_64-linux-gnu/5/libasan.so+0x373ef)
...
comment:6 by , 7 years ago
Apparently, the best way to solve this would be simply to disable instrumentation for this case:
#if defined(__clang__) || defined (__GNUC__)
# define ATTRIBUTE_NO_SANITIZE_ADDRESS __attribute__((no_sanitize_address))
#else
# define ATTRIBUTE_NO_SANITIZE_ADDRESS
#endif
...
ATTRIBUTE_NO_SANITIZE_ADDRESS
void ThisFunctionWillNotBeInstrumented() {...}
Unfortunately, I wasn't able to find the combination of methods I need to mark so far to make it work :-/
comment:7 by , 7 years ago
| Resolution: | invalid |
|---|---|
| Status: | closed → reopened |
comment:8 by , 7 years ago
Okay, an update on this ticket: I have finally managed to have another look at it, and here are the conclusions:
1) I was wrong to say that ramey's fix (see comment:5) does not resolve the original problem of the reporter (as in the UBSan still complains).
It does work indeed for me now on GCC 5.3 with UBSan enabled, it's just that after fixing the reported problem, the program would still crash if ASan is also enabled due to a different problem.
2) The source of the second problem is in void_caster and it is actually obvious if one attentively reads the backtrace from ASan and the code.
ASan doesn't like accesses to unknown addresses: AddressSanitizer: SEGV on unknown address 0x000000000008. This is caused by the constructor called just before the use() function and the following patch makes the problem go away:
--- boost/serialization/void_cast.hpp 2016-01-27 14:46:23.000000000 +0100
+++ boost/serialization/void_cast.hpp.orig 2016-01-18 11:50:19.000000000 +0100
@@ -180,7 +180,14 @@
void_caster_primitive<Derived, Base>::void_caster_primitive() :
void_caster(
& type_info_implementation<Derived>::type::get_const_instance(),
- & type_info_implementation<Base>::type::get_const_instance()
+ & type_info_implementation<Base>::type::get_const_instance(),
+ // note:I wanted to displace from 0 here, but at least one compiler
+ // treated 0 by not shifting it at all.
+ reinterpret_cast<std::ptrdiff_t>(
+ static_cast<Derived *>(
+ reinterpret_cast<Base *>(8)
+ )
+ ) - 8
)
{
recursive_register();
After seeing this code, I thinks it's quite obvious where the magical 0x000000000008 comes from...
So, having this said, the original problem is indeed solved, and an additional problem is at least diagnosed. Shall we keep this ticket open for this additional problem, or you would rather like to close it and have me open a new ticket for this second issue?
Of course, the workaround above works for me right now, but it would be great to have a better solution out of the box.
Thanks for your work on boost::serialization!
comment:9 by , 6 years ago
I've just checked with clang++-3.9 nightly + ASan, and, unfortunately, void_caster_primitive still causes a SEGV. Is there anything that can be done about it? The comment doesn't specify which compiler specifically was the problematic one and so I'm not sure whether just removing the code for GCC / Clang is "safe".
comment:10 by , 5 years ago
I am using boost 1.63.0, GCC 7.2.1, bintutils 2.29.1 and trying to compile software using boost with Undefined Behavior Sanitizer (UBSan). I think, am hitting the same issue.
Compile generates: mov (%r12),%rax and registers are:
rax 0x8 8 r12 0x8 8
Here is a full picture:
┌──/cvmfs/cms-ib.cern.ch/nweek-02500/slc6_amd64_gcc700/external/boost/1.63.0-mmelna2/include/boost/serialization/void_cast.hpp──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┐
│184 // note:I wanted to displace from 0 here, but at least one compiler │
│185 // treated 0 by not shifting it at all. │
│186 reinterpret_cast<std::ptrdiff_t>( │
>│187 static_cast<Derived *>( │
│188 reinterpret_cast<Base *>(8) │
│189 ) │
│190 ) - 8 │
│191 ) │
│192 { │
│193 recursive_register(); │
│194 } │
│195 │
│196 template <class Derived, class Base> │
│197 void_caster_primitive<Derived, Base>::~void_caster_primitive(){ │
│198 recursive_unregister(); │
│199 } │
│200 │
│201 template <class Derived, class Base> │
│202 class BOOST_SYMBOL_VISIBLE void_caster_virtual_base : │
│203 public void_caster │
│204 { │
┌───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┐
│0x7fffe6c4461c <boost::serialization::void_cast_detail::void_caster_primitive<cond::GenericSummary, cond::Summary>::void_caster_primitive()+138> mov $0x8,%r12d │
│0x7fffe6c44622 <boost::serialization::void_cast_detail::void_caster_primitive<cond::GenericSummary, cond::Summary>::void_caster_primitive()+144> mov %r12,%rax │
│0x7fffe6c44625 <boost::serialization::void_cast_detail::void_caster_primitive<cond::GenericSummary, cond::Summary>::void_caster_primitive()+147> test %r12,%r12 │
│0x7fffe6c44628 <boost::serialization::void_cast_detail::void_caster_primitive<cond::GenericSummary, cond::Summary>::void_caster_primitive()+150> je 0x7fffe6c44635 <boost::serialization::void_cast_detail::void_caster_primitive<cond::GenericSummary, cond::Su│
│0x7fffe6c4462a <boost::serialization::void_cast_detail::void_caster_primitive<cond::GenericSummary, cond::Summary>::void_caster_primitive()+152> mov %rax,%rdx │
│0x7fffe6c4462d <boost::serialization::void_cast_detail::void_caster_primitive<cond::GenericSummary, cond::Summary>::void_caster_primitive()+155> and $0x7,%edx │
│0x7fffe6c44630 <boost::serialization::void_cast_detail::void_caster_primitive<cond::GenericSummary, cond::Summary>::void_caster_primitive()+158> test %rdx,%rdx │
│0x7fffe6c44633 <boost::serialization::void_cast_detail::void_caster_primitive<cond::GenericSummary, cond::Summary>::void_caster_primitive()+161> je 0x7fffe6c44644 <boost::serialization::void_cast_detail::void_caster_primitive<cond::GenericSummary, cond::Su│
│0x7fffe6c44635 <boost::serialization::void_cast_detail::void_caster_primitive<cond::GenericSummary, cond::Summary>::void_caster_primitive()+163> mov %rax,%rsi │
│0x7fffe6c44638 <boost::serialization::void_cast_detail::void_caster_primitive<cond::GenericSummary, cond::Summary>::void_caster_primitive()+166> lea 0x15f221(%rip),%rdi # 0x7fffe6da3860 │
│0x7fffe6c4463f <boost::serialization::void_cast_detail::void_caster_primitive<cond::GenericSummary, cond::Summary>::void_caster_primitive()+173> callq 0x7fffe6bd73a0 <__ubsan_handle_type_mismatch@plt> │
>│0x7fffe6c44644 <boost::serialization::void_cast_detail::void_caster_primitive<cond::GenericSummary, cond::Summary>::void_caster_primitive()+178> mov (%r12),%rax │
│0x7fffe6c44648 <boost::serialization::void_cast_detail::void_caster_primitive<cond::GenericSummary, cond::Summary>::void_caster_primitive()+182> mov %rax,%rdx │
│0x7fffe6c4464b <boost::serialization::void_cast_detail::void_caster_primitive<cond::GenericSummary, cond::Summary>::void_caster_primitive()+185> movabs $0xd36d9746bc2d02eb,%rax │
│0x7fffe6c44655 <boost::serialization::void_cast_detail::void_caster_primitive<cond::GenericSummary, cond::Summary>::void_caster_primitive()+195> xor %rdx,%rax │
│0x7fffe6c44658 <boost::serialization::void_cast_detail::void_caster_primitive<cond::GenericSummary, cond::Summary>::void_caster_primitive()+198> mov %rax,%rcx │
│0x7fffe6c4465b <boost::serialization::void_cast_detail::void_caster_primitive<cond::GenericSummary, cond::Summary>::void_caster_primitive()+201> movabs $0x9ddfea08eb382d69,%rax │
│0x7fffe6c44665 <boost::serialization::void_cast_detail::void_caster_primitive<cond::GenericSummary, cond::Summary>::void_caster_primitive()+211> imul %rcx,%rax │
│0x7fffe6c44669 <boost::serialization::void_cast_detail::void_caster_primitive<cond::GenericSummary, cond::Summary>::void_caster_primitive()+215> mov %rax,%rcx │
│0x7fffe6c4466c <boost::serialization::void_cast_detail::void_caster_primitive<cond::GenericSummary, cond::Summary>::void_caster_primitive()+218> shl $0x2f,%rcx │
│0x7fffe6c44670 <boost::serialization::void_cast_detail::void_caster_primitive<cond::GenericSummary, cond::Summary>::void_caster_primitive()+222> xor %rdx,%rax │
└───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┘
multi-thre Thread 0x7ffff7dc97 In: boost::serialization::void_cast_detail::void_caster_primitive<cond::GenericSummary, cond::Summary>::void_caster_primitive L187 PC: 0x7fffe6c44644
Starting program: /cvmfs/cms-ib.cern.ch/nweek-02500/slc6_amd64_gcc700/cms/cmssw/CMSSW_10_0_UBSAN_X_2017-11-27-2300/external/slc6_amd64_gcc700/bin/python /cvmfs/cms-ib.cern.ch/week0/slc6_amd64_gcc700/cms/cmssw/CMSSW_10_0_UBSAN_X_2017-11-27-2300/src/FWCore/Utilities/scripts
/edmCheckClassVersion -l tmp/slc6_amd64_gcc700/src/CondFormats/Common/src/CondFormatsCommon/libCondFormatsCommon.so -x src/CondFormats/Common/src/classes_def.xml
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib64/libthread_db.so.1".
warning: File "/cvmfs/cms-ib.cern.ch/nweek-02500/slc6_amd64_gcc700/external/gcc/7.0.0-mmelna2/lib64/libstdc++.so.6.0.24-gdb.py" auto-loading has been declined by your `auto-load safe-path' set to "$debugdir:$datadir/auto-load".
Program received signal SIGSEGV, Segmentation fault.
0x00007fffe6c44644 in boost::serialization::void_cast_detail::void_caster_primitive<cond::GenericSummary, cond::Summary>::void_caster_primitive (
this=0x7fffe6e18a40 <boost::serialization::singleton<boost::serialization::void_cast_detail::void_caster_primitive<cond::GenericSummary, cond::Summary> >::get_instance()::t>)
at /cvmfs/cms-ib.cern.ch/nweek-02500/slc6_amd64_gcc700/external/boost/1.63.0-mmelna2/include/boost/serialization/void_cast.hpp:187
(gdb) p $_siginfo._sifields._sigfault.si_addr
$1 = (void *) 0x8
(gdb)
rax 0x8 8
rbx 0x7fffe6e18a40 140737066928704
rcx 0x14 20
rdx 0x0 0
rsi 0x12 18
rdi 0x7fffe6e18a40 140737066928704
rbp 0x7fffffff2660 0x7fffffff2660
rsp 0x7fffffff2640 0x7fffffff2640
r8 0x24ebc30 38714416
r9 0x6 6
r10 0x7fffffff23f0 140737488298992
r11 0x7fffe6c44592 140737065010578
r12 0x8 8
r13 0x7fffe6d02a98 140737065790104
r14 0x7fffffff3a20 140737488304672
r15 0x0 0
rip 0x7fffe6c44644 0x7fffe6c44644 <boost::serialization::void_cast_detail::void_caster_primitive<cond::GenericSummary, cond::Summary>::void_caster_primitive()+178>
eflags 0x10246 [ PF ZF IF RF ]
cs 0x33 51
ss 0x2b 43
ds 0x0 0
es 0x0 0
comment:11 by , 5 years ago
I must admit that after trying to fiddle with that function quite a bit, I resorted to just adding __attribute__ ((no_sanitize_undefined)) to void_caster_primitive<Derived, Base>::void_caster_primitive(). That shows at least that this in still a current problem.
Note:
See TracTickets
for help on using tickets.
I wasn't able to get the (address?) sanitizer working in my Xcode 6 environment. But I did trace through the program with the debugger. line 132 of singleton.hpp contains the line
where the value of instance has the (initialized value of zero). This might be considered an error by the sanitizer, but use(...) is an empty function. It is only included to guarantee that the singleton is called before main. If this isn't included, compile for release may optimize away the pre-main invocation and the class may not function as intended. So I would call this an over zealous behavior of the sanitizer. Or maybe the sanitizer could be considered not smart enough to trace one more level deep. or... what ever.