Boost C++ Libraries: Ticket #10899: symbol visibility: cannot catch an exception thrown by boost::throw_exception on mac OS https://svn.boost.org/trac10/ticket/10899 <h2 class="section" id="Theproblem">The problem</h2> <p> The problem showed up with an exception from boost::property_tree, but I believe it is more general. </p> <p> I have a library (named liba in the attached example) which </p> <ul><li>is built with hidden symbols visibility </li><li>uses boost::property_tree in a <code>deserialize()</code> function </li><li>catch <code>boost::property_tree::ptree_bad_path</code> exception if any (so this is not an "exception crossing dll boundary issue") </li></ul><p> This usually works (test_a works on all platforms) except when </p> <ul><li>we're on mac os </li><li>the <code>deserialize()</code> function is called from another library which also uses boost::property_tree </li></ul><p> In such a case, liba fails to catch the <code>boost::property_tree::ptree_bad_path</code> exception (test_b fails on mac os, but passes on linux). </p> <p> So, my problem is that on mac os, liba behaves differently depending on whether it's caller uses boost::property_tree or not. </p> <h2 class="section" id="Atentativeexplanation">A tentative explanation</h2> <p> I think the problem comes from symbol visibility inconsistency: </p> <ul><li>because of boost::throw_exception, the exception which is thrown is not a <code>boost::property_tree::ptree_bad_path</code> but a subclass <code>boost::exception_detail::error_info_injector&lt;boost::property_tree::ptree_bad_path&gt;</code> </li></ul><ul><li>in <code>include/boost/exception/exception.hpp</code>, <code>error_info_injector</code> visibility is forced to be public (this change comes from <a class="closed ticket" href="https://svn.boost.org/trac10/ticket/4594" title="#4594: Patches: Boost exception classes do not have GCC default visibility (closed: fixed)">#4594</a>) <pre class="wiki">#if defined(__GNUC__) # if (__GNUC__ == 4 &amp;&amp; __GNUC_MINOR__ &gt;= 1) || (__GNUC__ &gt; 4) # pragma GCC visibility push (default) # endif #endif template &lt;class T&gt; struct error_info_injector: public T, public exception { explicit error_info_injector( T const &amp; x ): T(x) { } ~error_info_injector() throw() { } }; </pre></li></ul><ul><li>However, <code>boost::property_tree::ptree_bad_path</code> is still hidden: <ul><li>there is no visibility #pragma in <code>include/boost/property_tree/exceptions.hpp</code> forcing it to be public </li><li>I build the library with hidden symbols by default </li><li>I want to catch the exception within the library, so I do not need to make the symbol public. </li></ul></li></ul><p> What seems to happen on mac (see the typeid adresses) when running test_b: </p> <ul><li>symbol <code>boost::exception_detail::error_info_injector&lt;boost::property_tree::ptree_bad_path&gt;</code> is global, and <em>the version from libb is used</em>. </li></ul><ul><li>symbol <code>boost::property_tree::ptree_bad_path</code> is private, each lib uses its own version </li></ul><ul><li>within liba, <code>boost::exception_detail::error_info_injector&lt;boost::property_tree::ptree_bad_path&gt;</code> is thrown, but liba fails to upcast it to its own version of <code>boost::property_tree::ptree_bad_path</code>, so it catches it as std::exception instead (and throw it again) </li></ul><ul><li>then within libb, the exception is caught again, but this time casting it to libb's version <code>boost::property_tree::ptree_bad_path</code> works. </li></ul><p> When calling the <code>serialize()</code> function from test_a (which does not use boost::property_tree), there is no symbol confusion and the exception is caught within liba, as expected. </p> <p> If this is right, the root cause is that one class is hidden and the other not. If we make both exception classes public or private, then it works as expected. </p> <p> I'm not sure why test_b passes on linux, it may have something to do with weak symbols: nm shows the public symbols as weak on linux, but not on mac. </p> <h2 class="section" id="Buildingandrunningonmac">Building and running on mac</h2> <pre class="wiki">$ make clean rm -f src/a.o src/liba.so src/test_a.o src/test_a src/b.o src/libb.so src/test_b.o src/test_b src/*.swp *.swp src/.a* src/.b* $ make all c++ -c -g -Iinclude -o src/test_a.o src/test_a.cc c++ -c -g -fPIC -Da_EXPORTS -Iinclude -I/Users/hcuche/.local/share/qi/toolchains/mac64/boost/include -o src/a.o src/a.cc c++ -shared -Wl -o src/liba.so src/a.o clang: warning: unknown warning option '-Wl' c++ -Lsrc -o src/test_a src/test_a.o -la c++ -c -g -Iinclude -o src/test_b.o src/test_b.cc c++ -c -g -fPIC -fvisibility=hidden -Db_EXPORTS -Iinclude -I/Users/hcuche/.local/share/qi/toolchains/mac64/boost/include -o src/b.o src/b.cc c++ -shared -Wl -o src/libb.so src/b.o src/liba.so clang: warning: unknown warning option '-Wl' c++ -Lsrc -o src/test_b src/test_b.o -lb -la $ DYLD_LIBRARY_PATH=./src ./src/test_a A::deserialize typeid(boost::property_tree::ptree_bad_path): 0x1085f2ed0 A::deserialize typeid(boost::exception_detail::error_info_injector&lt;boost::property_tree::ptree_bad_path&gt;): 0x1085f2f00 A::deserialize caught ptree_bad_path with typeid 0x1085f32d0 test_a: OK got "A::deserialize caught ptree_bad_path" $ DYLD_LIBRARY_PATH=./src ./src/test_b B::load typeid(boost::property_tree::ptree_bad_path): 0x10515f3d0 B::load typeid(boost::exception_detail::error_info_injector&lt;boost::property_tree::ptree_bad_path&gt;): 0x10515f400 A::deserialize typeid(boost::property_tree::ptree_bad_path): 0x105197ed0 A::deserialize typeid(boost::exception_detail::error_info_injector&lt;boost::property_tree::ptree_bad_path&gt;): 0x10515f400 A::deserialize caught std::exception with typeid 0x1051982d0 A::load caught ptree_bad_path with typeid 0x1051982d0 test_b: KO got "B::load caught ptree_bad_path" $ c++ --version Apple LLVM version 4.2 (clang-425.0.28) (based on LLVM 3.2svn) Target: x86_64-apple-darwin12.5.0 Thread model: posix </pre><h2 class="section" id="Buildingandrunningonlinuxwithgcc">Building and running on linux with gcc</h2> <pre class="wiki">$ make clean rm -f src/*.o src/*.so src/test_a src/test_b src/.a* src/.b* ._* src/._* include/._* $ make all g++ -c -g -Iinclude -o src/test_a.o src/test_a.cc g++ -c -g -fPIC -Da_EXPORTS -Iinclude -I/home/sbarthelemy/.local/share/qi/toolchains/linux64/boost/include -o src/a.o src/a.cc g++ -shared -Wl -o src/liba.so src/a.o g++ -Lsrc -o src/test_a src/test_a.o -la g++ -c -g -Iinclude -o src/test_b.o src/test_b.cc g++ -c -g -fPIC -fvisibility=hidden -Db_EXPORTS -Iinclude -I/home/sbarthelemy/.local/share/qi/toolchains/linux64/boost/include -o src/b.o src/b.cc g++ -shared -Wl -o src/libb.so src/b.o src/liba.so g++ -Lsrc -o src/test_b src/test_b.o -lb -la $ LD_LIBRARY_PATH=./src ./src/test_a A::deserialize typeid(boost::property_tree::ptree_bad_path): 0x7f1912652c60 A::deserialize typeid(boost::exception_detail::error_info_injector&lt;boost::property_tree::ptree_bad_path&gt;): 0x7f1912652c20 A::deserialize caught ptree_bad_path with typeid 0x7f1912652b80 test_a: OK got "A::deserialize caught ptree_bad_path" $ LD_LIBRARY_PATH=./src ./src/test_b B::load typeid(boost::property_tree::ptree_bad_path): 0x7fefc97dfce0 B::load typeid(boost::exception_detail::error_info_injector&lt;boost::property_tree::ptree_bad_path&gt;): 0x7fefc97dfca0 A::deserialize typeid(boost::property_tree::ptree_bad_path): 0x7fefc95cdc60 A::deserialize typeid(boost::exception_detail::error_info_injector&lt;boost::property_tree::ptree_bad_path&gt;): 0x7fefc97dfca0 A::deserialize caught ptree_bad_path with typeid 0x7fefc95cdb80 test_b: OK got "A::deserialize caught ptree_bad_path" $ g++ --version g++ (Ubuntu/Linaro 4.6.3-1ubuntu5) 4.6.3 Copyright (C) 2011 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. </pre><h2 class="section" id="Buildingandrunningonlinuxwithclang">Building and running on linux with clang</h2> <pre class="wiki">$ make clean rm -f src/*.o src/*.so src/test_a src/test_b src/.a* src/.b* ._* src/._* include/._* $ CXX=clang++ make all clang++ -c -g -Iinclude -o src/test_a.o src/test_a.cc clang++ -c -g -fPIC -Da_EXPORTS -Iinclude -I/home/sbarthelemy/.local/share/qi/toolchains/linux64/boost/include -o src/a.o src/a.cc clang++ -shared -Wl -o src/liba.so src/a.o clang++ -Lsrc -o src/test_a src/test_a.o -la clang++ -c -g -Iinclude -o src/test_b.o src/test_b.cc clang++ -c -g -fPIC -fvisibility=hidden -Db_EXPORTS -Iinclude -I/home/sbarthelemy/.local/share/qi/toolchains/linux64/boost/include -o src/b.o src/b.cc clang++ -shared -Wl -o src/libb.so src/b.o src/liba.so clang++ -Lsrc -o src/test_b src/test_b.o -lb -la $ LD_LIBRARY_PATH=./src ./src/test_a A::deserialize typeid(boost::property_tree::ptree_bad_path): 0x7f8bfa09e800 A::deserialize typeid(boost::exception_detail::error_info_injector&lt;boost::property_tree::ptree_bad_path&gt;): 0x7f8bfa09e830 A::deserialize caught ptree_bad_path with typeid 0x7f8bfa09ec00 test_a: OK got "A::deserialize caught ptree_bad_path" $ LD_LIBRARY_PATH=./src ./src/test_b B::load typeid(boost::property_tree::ptree_bad_path): 0x7f1d50790920 B::load typeid(boost::exception_detail::error_info_injector&lt;boost::property_tree::ptree_bad_path&gt;): 0x7f1d50790950 A::deserialize typeid(boost::property_tree::ptree_bad_path): 0x7f1d5057d800 A::deserialize typeid(boost::exception_detail::error_info_injector&lt;boost::property_tree::ptree_bad_path&gt;): 0x7f1d5057d830 A::deserialize caught ptree_bad_path with typeid 0x7f1d5057dc00 test_b: OK got "A::deserialize caught ptree_bad_path" $ clang++ --version Ubuntu clang version 3.3-5ubuntu4~precise1 (branches/release_33) (based on LLVM 3.3) Target: x86_64-pc-linux-gnu Thread model: posix </pre> en-us Boost C++ Libraries /htdocs/site/boost.png https://svn.boost.org/trac10/ticket/10899 Trac 1.4.3 Sébastien Barthélémy <barthelemy@…> Fri, 26 Dec 2014 12:57:08 GMT attachment set https://svn.boost.org/trac10/ticket/10899 https://svn.boost.org/trac10/ticket/10899 <ul> <li><strong>attachment</strong> → <span class="trac-field-new">visibilibug.tgz</span> </li> </ul> <p> minimalistic example </p> Ticket Emil Dotchevski Sat, 27 Dec 2014 21:18:16 GMT <link>https://svn.boost.org/trac10/ticket/10899#comment:1 </link> <guid isPermaLink="false">https://svn.boost.org/trac10/ticket/10899#comment:1</guid> <description> <p> I left my mac at the office so I can't run your test right now, but consider that on MSVC the types of exception objects are compared by a "strcmp" of the mangled names from their typeinfo. On GCC (probably on clang too) what's compared is the address of the typeinfos. That's why the visibility matters: when binaries are linked (dynamically or statically), unless two typeinfos get recognized as the same type by the linker, they won't match in a catch statement. Since each catch statement uses exactly one of the possibly many typeinfo objects that exist for the same type, depending on where the exception object originated, it might catch it -- or not. </p> <p> Things are even more complicated when virtual inheritance is used in exception type hierarchies, which is of course a "best practice". In that case the catch lookup has to consider a whole list of typeinfos, and if even one of them is different, the exception won't get recognized. </p> <p> P.S. I don't think that your problem has anything to do with the mechanics of boost::throw_exception, but if you feel otherwise you might try to edit that out of your test to verify that. </p> </description> <category>Ticket</category> </item> <item> <author>Sébastien Barthélémy <barthelemy@…></author> <pubDate>Mon, 02 Feb 2015 15:03:38 GMT</pubDate> <title/> <link>https://svn.boost.org/trac10/ticket/10899#comment:2 </link> <guid isPermaLink="false">https://svn.boost.org/trac10/ticket/10899#comment:2</guid> <description> <p> Hello Emil, </p> <p> thank you very much for your answer (and sorry for the late reply). </p> <p> Replying to <a class="ticket" href="https://svn.boost.org/trac10/ticket/10899#comment:1" title="Comment 1">emildotchevski</a>: </p> <blockquote class="citation"> <p> consider that on MSVC the types of exception objects are compared by a "strcmp" of the mangled names from their typeinfo. On GCC (probably on clang too) what's compared is the address of the typeinfos. </p> </blockquote> <p> See the link below, apparently GCC aligned with MSVC. I don't know about clang, and more specifically about clang on mac os. </p> <p> <a class="ext-link" href="http://stackoverflow.com/questions/14268736/symbol-visibility-exceptions-runtime-error"><span class="icon">​</span>http://stackoverflow.com/questions/14268736/symbol-visibility-exceptions-runtime-error</a> </p> <blockquote class="citation"> <p> That's why the visibility matters: when binaries are linked (dynamically or statically), unless two typeinfos get recognized as the same type by the linker, they won't match in a catch statement. Since each catch statement uses exactly one of the possibly many typeinfo objects that exist for the same type, depending on where the exception object originated, it might catch it -- or not. </p> </blockquote> <p> Ok, I have the same understanding of the issue, thank you for the confirmation. </p> <blockquote class="citation"> <p> P.S. I don't think that your problem has anything to do with the mechanics of boost::throw_exception, &gt; but if you feel otherwise you might try to edit that out of your test to verify that. </p> </blockquote> <p> As I explained in the bug report: </p> <p> we tried to patch boost to make boost::exception_detail::error_info_injector&lt;boost::property_tree::ptree_bad_path&gt; visible, and it did fix the issue. </p> <p> we also tried to patch boost to make boost::property_tree::ptree_bad_path hidden (removing <code>#pragma GCC visibility push (default)</code>) and it also did fix the issue. </p> <p> As a consequence, I do think it is releated to the visibility of the templated classes boost::throw_exception is creating. </p> </description> <category>Ticket</category> </item> </channel> </rss>