Python GIL not acquired before freeing shared_ptr, causes crash

[team@…]

We're successfully using boost::python with Python on multiple threads that lock and release the GIL as appropriate, and correctly set the Python thread state, etc.

One thing we don't have control over, however, is the freeing of shared_ptr objects which were created by boost::python. This may happen on a different thread than the previous Python execution and/or without the GIL being locked. To fix this, the following patch is required:

--- builtin_converters.cpp	2011-06-07 06:15:33.000000000 +0200
+++ builtin_converters.cpp.fixed	2013-03-13 12:57:29.346638173 +0100
@@ -32,7 +32,9 @@
 
 void shared_ptr_deleter::operator()(void const*)
 {
+    PyGILState_STATE gil = PyGILState_Ensure();
     owner.reset();
+    PyGILState_Release(gil);
 }
 
 namespace

This fix can be applied externally to boost::python, but requires a few header files that re-implement shared_ptr_to_python.hpp and shared_ptr_from_python.hpp.

[eu@…]

I am bumping into this issue, having crash with this traceback

  boost::python::converter::shared_ptr_deleter::operator()(void const*) () // CRASH here
  boost::detail::sp_counted_base::release 
  ~shared_count 

It is a python-instantiated object which is deleted in c++ because its refcount drops to zero. I also confirm that locking the GIL solves the issue thanks for the tip). Would you mind sharing your workaround?

Cheers!

[eu@…]

(The workaround I mean is how to apply this to boost::python externally -- I need to support linking against officially compiled libs.)

[Dane Springmeyer]

Any chance this will be applied? I'm also seeing this crash in Mapnik.

[anonymous]

Please, can someone apply this patch?

[anonymous]

Any change someone can look at and apply this patch?

[anonymous]

Could you please try the Python C++-SIG to get help? I don' know if this change is safe. Sorry.

[leaf.nunes@…]

This seems to be a bigger problem with python 2.7; is there an objection to this patch? My site is having to re-apply a different patch to work around the same issue (every time we install a new version of boost).

[luke.titley@…]

We're having to hack around this too. It's a problem with boost::python not python.

[anonymous]

I posted to c++-sig in May, those folks are afraid that it might cause deadlocks ​http://code.activestate.com/lists/python-cplusplus-sig/17196/ . Please someone else post a follow-up there, this Trac is abandoned (we can move to github if someone opens an issue in ​https://github.com/boostorg/python/issues).

[anonymous]

PS there is a pull request to fix it (with an interesting discussion) at ​https://github.com/boostorg/python/pull/11 .

[luke.titley@…]

Thanks for this. For now we've patched boost python.