Missing Python numeric references on Windows

[bsherwood]

In Boost 1.33.1, needed references to the Python
numeric module are missing when Boost is compiled on
Windows under Msys using MinGW. A colleague gave me the
following arcane workaround:

> cd
bin/boost/libs/python/build/boost_python.dll/mingw/release/threading-multi/

Correct an error in the boost library, which is missing
references to the numeric module:

> g++ -shared -o boost_python-mgw-mt-1_33_1.dll *.obj
-Wl,--out-implib,boost_python-mgw-mt-1_33_1.lib
/c/WINDOWS/system32/python24.dll 

[david_abrahams]

Logged In: YES 
user_id=52572

This report doesn't describe the symptoms of the problem
(what goes wrong) or the expected correct behavior, and it
doesn't say what the workaround does or in which directory
it should be executed.  As far as I can tell the workaround
isn't doing anything with "the numeric module" (whatever
that is).  So I'd need a lot more specific information
before I'd be able to start thinking about fixes.  Like,
what's the problem? ;-)

[bsherwood]

Logged In: YES 
user_id=34881

Okay, here's more detail: On Windows under Msys using MinGW
I try to compile the Visual module for Python (see
vpython.org). Visual is written in C++. With Boost 1.33.1,
but not with some earlier versions of Boost, the build fails
due to missing numeric references. Jonathan Brandmeyer gave
me the arcane workaround, which alas I don't understand.
Note that I did specify the directory in which to execute
the statement. If you examine the Boost library (before
using the workaround) you'll see that Numeric entries aren't
there, and after running the workaround they are there. What
additional specific information is needed?

[david_abrahams]

Logged In: YES 
user_id=52572

I'd like to know, specifically, what "missing numeric
references" means.

[nobody]

Logged In: NO 

Alas, I didn't save a listing of the library contents before
and after. But if you examine the library before and after
applying the workaround, you'll see that a huge number
(all?) of the numeric references are missing from the
defective library.

I'm sorry if this is insufficient information, and
unfortunately I'm currently deeply involved in a textbook
revision so I can't go start over and provide chapter and
verse. I guess I'll have to return to this in a couple of
months and give you the full documentation you need.

Remember that a functional description of the problem is
that I tried to build the Visual module (called visualpython
at sourceforge), which is written in C++, on MSYS with
MinGW, and I couldn't link because of a large number of
missing numeric references. The workaround got those
references into the library and I was able to finish the build.

[bsherwood]

Logged In: YES 
user_id=34881

Oops. Sorry to have failed to log in. That last comment was
from Bruce Sherwood.

[david_abrahams]

Logged In: YES 
user_id=52572

Bruce, the problem here is that I asked you to define a
term: "missing numeric references."  Instead of telling me
what that means, you gave me a description of your linking
problem again using the very term I asked you to define.  

Please tell me what a "missing numeric reference" is.  

For example, does it mean that there are symbols from the
NumPy or NumArray packages that the linker complains are
missing?  Or is it something else?  If so, you can start by
being more specific about what you mean by "numeric."

For what it's worth, the Boost.Python compiled library isn't
supposed to contain any references to the NumPy or NumArray
libraries.  If it did, nobody would be able to build
Boost.Python without a library that's not part of a stock
Python installation.

[bsherwood]

Logged In: YES 
user_id=34881

Sorry to be failing to communicate well; I'm a bit at sea on
this. The linker complains that Numeric symbols are missing.
Here is I hope a more operational definition of the problem.

I'm specifically referring to the old Numeric module, which
preceded NumArray.

I build Boost with this:
bjam -sTOOLS=mingw -sPYTHON_ROOT=/c/Python24
-sPYTHON_VERSION=2.4 stage

> cd
bin/boost/libs/python/build/boost_python.dll/mingw/release/threading-multi/
If you execute "nm boost_python-mgw-mt-1_33_1.lib >
nm-lib.txt" and inspect
the output file, you will find there are no mentions of
numeric, which leads
to failure of the build of Visual for VPython.
		
To get the references to the numeric module into the library:
> g++ -shared -o boost_python-mgw-mt-1_33_1.dll *.obj
-Wl,--out-implib,boost_python-mgw-mt-1_33_1.lib
/c/WINDOWS/system32/python24.dll 
> cp boost_python-mgw-mt-1_33_1.dll /mingw/bin
> cp boost_python-mgw-mt-1_33_1.lib
/mingw/lib/libboost_python24.a 
If you again run "nm" you'll now find lots of mentions of
numeric, such as

d000267.o:
00000000 i .idata$4
00000000 i .idata$5
00000000 i .idata$6
00000000 i .idata$7
00000000 t .text
         U __head_boost_python_mgw_mt_1_33_1_dll
00000000 I
__imp___ZN5boost6python7numeric3aux10array_baseC2ERKNS0_3api6objectES7_S7_S7_
00000000 T
__ZN5boost6python7numeric3aux10array_baseC2ERKNS0_3api6objectES7_S7_S7_

With earlier versions of Boost I didn't have to make any
adjustment in order to build Visual.Is there something about
1.33 that requires me to do something different in building
Boost than I did before?

[david_abrahams]

Logged In: YES 
user_id=52572

I think you're confused about the meaning of the "numeric"
symbols you were seeing, which made it very hard for me to
understand your problem.   The numeric symbols you mention
have (almost) nothing to do with the Python Numeric module:
they're symbols in the Boost.Python library.  In fact,
__imp___ZN5boost6python7numeric3aux10array_baseC2ERKNS0_3api6objectES7_S7_S7
corresponds to something like 

boost::python::numeric::array_base::array_base(boost::python::api::object
const&, boost::python::api::object const&,
boost::python::api::object const&,
boost::python::api::object const&,
boost::python::api::object const&,
boost::python::api::object const&);

The incantation you corrects the error looks to me just like
what I'd expect our build system to do as the final step of
building boost_python-mgw-mt-1_33_1.dll.  So now I'm
wondering how you came up with the version of the library
you say needed correction.  

If you used bjam to build boost, could you invoke it again
with "-n -a" in the command line, capture the output in a
file, and grep for boost_python-mgw-mt-1_33_1.dll ?

That would show us the command line that the build system is
/actually/ using to assemble the library.

Thanks,
Dave
   

[david_abrahams]

Logged In: YES 
user_id=52572

Not having heard any reply, I'm going to close this one. 
I'm sorry to do that because it seemed like we were getting
so close to an answer, but I can't get to the end without
your help.  Please feel free to reopen this issue if you
care to follow up.

[bsherwood]

Logged In: YES 
user_id=34881

Sorry, but a combination of computer problems and other work
prevented me from running your test until now. 

I started fresh: unpacked the tar file and executed in MSYS
on Windows the following:

bjam -n -a -sTOOLS=mingw -sPYTHON_ROOT=/c/Python24
-sPYTHON_VERSION=2.4 stage > save.txt

I then grepped for boost_python-mgw-mt-1_33_1.dll, and I
attach a file with the output. Hope this helps.