symbol visibility: cannot catch an exception thrown by boost::throw_exception on mac OS

[Sébastien Barthélémy <barthelemy@…>]

The problem

The problem showed up with an exception from boost::property_tree, but I believe it is more general.

I have a library (named liba in the attached example) which

• is built with hidden symbols visibility
• uses boost::property_tree in a deserialize() function
• catch boost::property_tree::ptree_bad_path exception if any (so this is not an "exception crossing dll boundary issue")

This usually works (test_a works on all platforms) except when

• we're on mac os
• the deserialize() function is called from another library which also uses boost::property_tree

In such a case, liba fails to catch the boost::property_tree::ptree_bad_path exception (test_b fails on mac os, but passes on linux).

So, my problem is that on mac os, liba behaves differently depending on whether it's caller uses boost::property_tree or not.

A tentative explanation

I think the problem comes from symbol visibility inconsistency:

• because of boost::throw_exception, the exception which is thrown is not a boost::property_tree::ptree_bad_path but a subclass boost::exception_detail::error_info_injector<boost::property_tree::ptree_bad_path>

• in include/boost/exception/exception.hpp, error_info_injector visibility is forced to be public (this change comes from #4594)

  #if defined(__GNUC__)
  # if (__GNUC__ == 4 && __GNUC_MINOR__ >= 1) || (__GNUC__ > 4)
  #  pragma GCC visibility push (default)
  # endif
  #endif
          template <class T>
          struct
          error_info_injector:
              public T,
              public exception
              {
              explicit
              error_info_injector( T const & x ):
                  T(x)
                  {
                  }
  
              ~error_info_injector() throw()
                  {
                  }
              };
  

• However, boost::property_tree::ptree_bad_path is still hidden:
  • there is no visibility #pragma in include/boost/property_tree/exceptions.hpp forcing it to be public
  • I build the library with hidden symbols by default
  • I want to catch the exception within the library, so I do not need to make the symbol public.

What seems to happen on mac (see the typeid adresses) when running test_b:

• symbol boost::exception_detail::error_info_injector<boost::property_tree::ptree_bad_path> is global, and the version from libb is used.

• symbol boost::property_tree::ptree_bad_path is private, each lib uses its own version

• within liba, boost::exception_detail::error_info_injector<boost::property_tree::ptree_bad_path> is thrown, but liba fails to upcast it to its own version of boost::property_tree::ptree_bad_path, so it catches it as std::exception instead (and throw it again)

• then within libb, the exception is caught again, but this time casting it to libb's version boost::property_tree::ptree_bad_path works.

When calling the serialize() function from test_a (which does not use boost::property_tree), there is no symbol confusion and the exception is caught within liba, as expected.

If this is right, the root cause is that one class is hidden and the other not. If we make both exception classes public or private, then it works as expected.

I'm not sure why test_b passes on linux, it may have something to do with weak symbols: nm shows the public symbols as weak on linux, but not on mac.

Building and running on mac

$ make clean
rm -f src/a.o src/liba.so src/test_a.o src/test_a src/b.o src/libb.so src/test_b.o src/test_b src/*.swp *.swp src/.a* src/.b*

$ make all
c++ -c -g -Iinclude -o src/test_a.o src/test_a.cc
c++ -c -g -fPIC -Da_EXPORTS -Iinclude -I/Users/hcuche/.local/share/qi/toolchains/mac64/boost/include -o src/a.o src/a.cc
c++ -shared -Wl -o src/liba.so src/a.o
clang: warning: unknown warning option '-Wl'
c++  -Lsrc -o src/test_a src/test_a.o -la
c++ -c -g -Iinclude -o src/test_b.o src/test_b.cc
c++ -c -g -fPIC -fvisibility=hidden -Db_EXPORTS -Iinclude -I/Users/hcuche/.local/share/qi/toolchains/mac64/boost/include -o src/b.o src/b.cc
c++ -shared -Wl -o src/libb.so src/b.o src/liba.so
clang: warning: unknown warning option '-Wl'
c++ -Lsrc -o src/test_b src/test_b.o -lb -la

$ DYLD_LIBRARY_PATH=./src ./src/test_a
A::deserialize typeid(boost::property_tree::ptree_bad_path): 0x1085f2ed0
A::deserialize typeid(boost::exception_detail::error_info_injector<boost::property_tree::ptree_bad_path>): 0x1085f2f00
A::deserialize caught ptree_bad_path with typeid 0x1085f32d0
test_a: OK got "A::deserialize caught ptree_bad_path"

$ DYLD_LIBRARY_PATH=./src ./src/test_b                                                                        B::load typeid(boost::property_tree::ptree_bad_path): 0x10515f3d0
B::load typeid(boost::exception_detail::error_info_injector<boost::property_tree::ptree_bad_path>): 0x10515f400
A::deserialize typeid(boost::property_tree::ptree_bad_path): 0x105197ed0
A::deserialize typeid(boost::exception_detail::error_info_injector<boost::property_tree::ptree_bad_path>): 0x10515f400
A::deserialize caught std::exception with typeid 0x1051982d0
A::load caught ptree_bad_path with typeid 0x1051982d0
test_b: KO got "B::load caught ptree_bad_path"

$ c++ --version                                                                                                                                                                                                                                                                                          
Apple LLVM version 4.2 (clang-425.0.28) (based on LLVM 3.2svn)
Target: x86_64-apple-darwin12.5.0
Thread model: posix

Building and running on linux with gcc

$ make clean
rm -f src/*.o src/*.so src/test_a src/test_b src/.a* src/.b* ._* src/._* include/._*

$ make all
g++ -c -g -Iinclude -o src/test_a.o src/test_a.cc
g++ -c -g -fPIC -Da_EXPORTS -Iinclude -I/home/sbarthelemy/.local/share/qi/toolchains/linux64/boost/include -o src/a.o src/a.cc
g++ -shared -Wl -o src/liba.so src/a.o
g++  -Lsrc -o src/test_a src/test_a.o -la
g++ -c -g -Iinclude -o src/test_b.o src/test_b.cc
g++ -c -g -fPIC -fvisibility=hidden -Db_EXPORTS -Iinclude -I/home/sbarthelemy/.local/share/qi/toolchains/linux64/boost/include -o src/b.o src/b.cc
g++ -shared -Wl -o src/libb.so src/b.o src/liba.so
g++ -Lsrc -o src/test_b src/test_b.o -lb -la

$ LD_LIBRARY_PATH=./src ./src/test_a
A::deserialize typeid(boost::property_tree::ptree_bad_path): 0x7f1912652c60
A::deserialize typeid(boost::exception_detail::error_info_injector<boost::property_tree::ptree_bad_path>): 0x7f1912652c20
A::deserialize caught ptree_bad_path with typeid 0x7f1912652b80
test_a: OK got "A::deserialize caught ptree_bad_path"

$ LD_LIBRARY_PATH=./src ./src/test_b
B::load typeid(boost::property_tree::ptree_bad_path): 0x7fefc97dfce0
B::load typeid(boost::exception_detail::error_info_injector<boost::property_tree::ptree_bad_path>): 0x7fefc97dfca0
A::deserialize typeid(boost::property_tree::ptree_bad_path): 0x7fefc95cdc60
A::deserialize typeid(boost::exception_detail::error_info_injector<boost::property_tree::ptree_bad_path>): 0x7fefc97dfca0
A::deserialize caught ptree_bad_path with typeid 0x7fefc95cdb80
test_b: OK got "A::deserialize caught ptree_bad_path"

$ g++ --version
g++ (Ubuntu/Linaro 4.6.3-1ubuntu5) 4.6.3
Copyright (C) 2011 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

Building and running on linux with clang

$ make clean
rm -f src/*.o src/*.so src/test_a src/test_b src/.a* src/.b* ._* src/._* include/._*

$ CXX=clang++ make all
clang++ -c -g -Iinclude -o src/test_a.o src/test_a.cc
clang++ -c -g -fPIC -Da_EXPORTS -Iinclude -I/home/sbarthelemy/.local/share/qi/toolchains/linux64/boost/include -o src/a.o src/a.cc
clang++ -shared -Wl -o src/liba.so src/a.o
clang++  -Lsrc -o src/test_a src/test_a.o -la
clang++ -c -g -Iinclude -o src/test_b.o src/test_b.cc
clang++ -c -g -fPIC -fvisibility=hidden -Db_EXPORTS -Iinclude -I/home/sbarthelemy/.local/share/qi/toolchains/linux64/boost/include -o src/b.o src/b.cc
clang++ -shared -Wl -o src/libb.so src/b.o src/liba.so
clang++ -Lsrc -o src/test_b src/test_b.o -lb -la

$ LD_LIBRARY_PATH=./src ./src/test_a
A::deserialize typeid(boost::property_tree::ptree_bad_path): 0x7f8bfa09e800
A::deserialize typeid(boost::exception_detail::error_info_injector<boost::property_tree::ptree_bad_path>): 0x7f8bfa09e830
A::deserialize caught ptree_bad_path with typeid 0x7f8bfa09ec00
test_a: OK got "A::deserialize caught ptree_bad_path"

$ LD_LIBRARY_PATH=./src ./src/test_b
B::load typeid(boost::property_tree::ptree_bad_path): 0x7f1d50790920
B::load typeid(boost::exception_detail::error_info_injector<boost::property_tree::ptree_bad_path>): 0x7f1d50790950
A::deserialize typeid(boost::property_tree::ptree_bad_path): 0x7f1d5057d800
A::deserialize typeid(boost::exception_detail::error_info_injector<boost::property_tree::ptree_bad_path>): 0x7f1d5057d830
A::deserialize caught ptree_bad_path with typeid 0x7f1d5057dc00
test_b: OK got "A::deserialize caught ptree_bad_path"

$ clang++ --version
Ubuntu clang version 3.3-5ubuntu4~precise1 (branches/release_33) (based on LLVM 3.3)
Target: x86_64-pc-linux-gnu
Thread model: posix

[Sébastien Barthélémy <barthelemy@…>]

minimalistic example

[Emil Dotchevski]

I left my mac at the office so I can't run your test right now, but consider that on MSVC the types of exception objects are compared by a "strcmp" of the mangled names from their typeinfo. On GCC (probably on clang too) what's compared is the address of the typeinfos. That's why the visibility matters: when binaries are linked (dynamically or statically), unless two typeinfos get recognized as the same type by the linker, they won't match in a catch statement. Since each catch statement uses exactly one of the possibly many typeinfo objects that exist for the same type, depending on where the exception object originated, it might catch it -- or not.

Things are even more complicated when virtual inheritance is used in exception type hierarchies, which is of course a "best practice". In that case the catch lookup has to consider a whole list of typeinfos, and if even one of them is different, the exception won't get recognized.

P.S. I don't think that your problem has anything to do with the mechanics of boost::throw_exception, but if you feel otherwise you might try to edit that out of your test to verify that.

[Sébastien Barthélémy <barthelemy@…>]

Hello Emil,

thank you very much for your answer (and sorry for the late reply).

Replying to emildotchevski:

consider that on MSVC the types of exception objects are compared by a "strcmp" of the mangled names from their typeinfo. On GCC (probably on clang too) what's compared is the address of the typeinfos.

See the link below, apparently GCC aligned with MSVC. I don't know about clang, and more specifically about clang on mac os.

​http://stackoverflow.com/questions/14268736/symbol-visibility-exceptions-runtime-error

That's why the visibility matters: when binaries are linked (dynamically or statically), unless two typeinfos get recognized as the same type by the linker, they won't match in a catch statement. Since each catch statement uses exactly one of the possibly many typeinfo objects that exist for the same type, depending on where the exception object originated, it might catch it -- or not.

Ok, I have the same understanding of the issue, thank you for the confirmation.

P.S. I don't think that your problem has anything to do with the mechanics of boost::throw_exception, > but if you feel otherwise you might try to edit that out of your test to verify that.

As I explained in the bug report:

we tried to patch boost to make boost::exception_detail::error_info_injector<boost::property_tree::ptree_bad_path> visible, and it did fix the issue.

we also tried to patch boost to make boost::property_tree::ptree_bad_path hidden (removing #pragma GCC visibility push (default)) and it also did fix the issue.

As a consequence, I do think it is releated to the visibility of the templated classes boost::throw_exception is creating.