Boost.MPI Compile failure with Python 3

[smr@…]

Boost 1.44 configured to use Python 3.1 fails on two Boost.MPI files. The two fatal error messages are reproduced below, edited (see ​http://lists.boost.org/Archives/boost/2010/09/170659.php for the full error output).

This failure was discussed in Debian (​http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=595786) and Andreas Kloeckner provided a patch. See attached.

-Steve

"g++" -ftemplate-depth-128 -O3 -finline-functions -Wno-inline -Wall -g -D_REENTRANT -pthread -fPIC -DBOOST_ALL_NO_LIB=1 -DBOOST_MPI_DYN_LINK=1 -DBOOST_MPI_PYTHON_DYN_LINK=1 -DBOOST_PYTHON_DYN_LINK=1 -DNDEBUG -I"." -I"/usr/include/python3.1" -I"/usr/lib/openmpi/include" -I"/usr/lib/openmpi/include/openmpi" -c -o "bin.v2/libs/mpi/build/gcc-4.4.5/release/debug-symbols-on/python-3.1/threading-multi/python/datatypes.o" "libs/mpi/src/python/datatypes.cpp"

libs/mpi/src/python/datatypes.cpp: In function ‘void boost::mpi::python::export_datatypes()’: libs/mpi/src/python/datatypes.cpp:20: error: ‘PyInt_Type’ was not declared in this scope In file included from ./boost/function/detail/prologue.hpp:17,

from ./boost/function/function_template.hpp:13, from ./boost/function/detail/maybe_include.hpp:13, from ./boost/function/function0.hpp:11, from ./boost/python/errors.hpp:13, from ./boost/python/handle.hpp:11, from ./boost/python/converter/arg_to_python_base.hpp:7, from ./boost/python/converter/arg_to_python.hpp:14, from ./boost/python/call.hpp:15, from ./boost/python/object_core.hpp:14, from ./boost/python/object.hpp:9, from ./boost/mpi/python/serialize.hpp:25, from libs/mpi/src/python/datatypes.cpp:13:

...failed gcc.compile.c++ bin.v2/libs/mpi/build/gcc-4.4.5/release/debug-symbols-on/python-3.1/threading-multi/python/datatypes.o...

and

"g++" -ftemplate-depth-128 -O3 -finline-functions -Wno-inline -Wall -g -D_REENTRANT -pthread -fPIC -DBOOST_ALL_NO_LIB=1 -DBOOST_MPI_DYN_LINK=1 -DBOOST_MPI_PYTHON_DYN_LINK=1 -DBOOST_PYTHON_DYN_LINK=1 -DNDEBUG -I"." -I"/usr/include/python3.1" -I"/usr/lib/openmpi/include" -I"/usr/lib/openmpi/include/openmpi" -c -o "bin.v2/libs/mpi/build/gcc-4.4.5/release/debug-symbols-on/python-3.1/threading-multi/python/py_environment.o" "libs/mpi/src/python/py_environment.cpp"

libs/mpi/src/python/py_environment.cpp: In function ‘bool boost::mpi::python::mpi_init(boost::python::list, bool)’: libs/mpi/src/python/py_environment.cpp:53: error: cannot convert ‘char’ to ‘wchar_t’ for argument ‘2’ to ‘void PySys_SetArgv(int, wchar_t)’ ...failed gcc.compile.c++ bin.v2/libs/mpi/build/gcc-4.4.5/release/debug-symbols-on/python-3.1/threading-multi/python/py_environment.o...

[Matthias Troyer]

What are the license conditions for the part inserted as " Code stolen from py3k/Modules/python.c."?

[inform@…]

​http://docs.python.org/py3k/license.html#terms-and-conditions-for-accessing-or-otherwise-using-python

Andreas

[anonymous]

Thanks! The only issue I see here is that they require that "PSF’s License Agreement " is retained, while we want to publish it under the Boost license. Do you know the authors of that piece of code so that we can ask them whether we can use it here? Otherwise we'll have to reimplement a similar idea but can't just copy&paste.

[inform@…]

Well, I think we can consider the patch a reimplementation, since I had to change their UCS handling functions to codecvt. I don't think it's likely that the Python guys will come suing us over this.

Andreas

[anonymous]

Thanks for the clarification.

[Tim Dawborn <tim.dawborn@…>]

This problem still exists in 1.46.1. Is there any chance of getting this patch pushed through into trunk?

[anonymous]

This problem still exists in 1.48.0

[anonymous]

Can you test whether the trunk works for you?

[smr@…]

I can confirm the trunk now works: I built against python 3.2. Thanks, -Steve

[smr@…]

Can this be pushed to release, please?

[anonymous]

The attached patch (as well as r76290) seems to introduce memory leak in argv conversion code. When you return false from the loop in bool mpi_init(list python_argv, bool abort_on_exception), memory allocated for argv_copy and argv_copy2 is not freed.

Also, the following code returns 1 in case of error, which is converted to true (because function is returning boolean):

if (myresult != std::codecvt<wchar_t,char,mbstate_t>::ok ) 
{ 
    fprintf(stderr, "failure translating argv\n"); 
    return 1; 
} 

[anonymous]

Still broken in 1.54 with Python 3.3.3.