Opened 12 years ago
Last modified 10 years ago
#4864 new Bugs
Support for 64-bit ICU
Reported by: | Dave Abrahams | Owned by: | Vladimir Prus |
---|---|---|---|
Milestone: | To Be Determined | Component: | build |
Version: | Boost 1.44.0 | Severity: | Problem |
Keywords: | Cc: | Katie Chan, cnewbold@…, ghost@…, dtrebbien@… |
Description
On windows, anyway, ICU's 64-bit binaries tend to appear in a subdirectory called lib64/
, but the Jamfile seems to assume that ICU will be in lib/
.
Attachments (1)
Change History (39)
comment:1 by , 12 years ago
comment:2 by , 12 years ago
Dave, can you tell me what's in has_icu.exe.rsp?
Also the bjam command line.
Thanks, John.
comment:3 by , 12 years ago
sorry, John... been up for 3 days trying to get all this building/working and I didn't take the time to create a little test case. The command-line was undoubtedly just something like bjam address-model=64
in libs/regex/test
. However, I don't have the other file. I could try to recreate this if you ping me in a couple of days.
comment:4 by , 12 years ago
There's a tentative fix for this in Trunk, if you can let me know if this works once you catch breath, that would be great.
comment:5 by , 12 years ago
Cc: | added |
---|
Nope, it completely broke things now. :D
Before, x64 actually worked if one renamed the directory from lib64 to lib. I had
D:\libs\icu4c-4_4_2\ // x86 D:\libs\icu4c-4_4_2-x64\ // x64
where both contained a \bin & \lib. i.e. I renamed the bin64 & lib64 and everything worked.
The new Jamfile.v2 causes ICU not to be detected at all, even for x86.
msvc.link bin.v2\libs\regex\build\msvc-10.0\debug\threading-multi\has_icu.exe LINK : fatal error LNK1181: cannot open input file 'icuucd.lib' call "D:\Program Files (x86)\Microsoft Visual Studio 10.0\VC\vcvarsall.bat" x86 >nul link /NOLOGO /INCREMENTAL:NO /DEBUG /MACHINE:X86 /subsystem:console /out:"bin.v2\libs\regex\build\msvc-10.0\debug\threading-multi\has_icu.exe" /delayload:icudt44.dll /delayload:icuin44.dll /delayload:icuin44d.dll /delayload:icuuc44.dll /delayload:icuuc44d.dll delayimp.lib @"bin.v2\libs\regex\build\msvc-10.0\debug\threading-multi\has_icu.exe.rsp" if %ERRORLEVEL% NEQ 0 EXIT %ERRORLEVEL% ...failed msvc.link bin.v2\libs\regex\build\msvc-10.0\debug\threading-multi\has_icu.exe bin.v2\libs\regex\build\msvc-10.0\debug\threading-multi\has_icu.pdb... ...failed updating 1 target... ...updated 8 targets... ...failed updating 1 target... Performing configuration checks - has_icu builds : no
comment:6 by , 12 years ago
The new Jamfile.v2 causes ICU not to be detected at all, even for x86.
Dang, it works for me here on Win32 (Vista)... two questions, what's the bjam invocation, and what's the content of bin.v2\libs\regex\build\msvc-10.0\debug\threading-multi\has_icu.exe.rsp?
Thanks, John.
comment:7 by , 12 years ago
Boost trunk r66651.
%ICU_PATH%=D:/libs/icu4c-4_4_2
%PATH%=....;%ICU_PATH%/bin
what's the bjam invocation
bjam --build-type=complete --with-regex stage
what's the content of bin.v2\libs\regex\build\msvc-10.0\debug\threading-multi\has_icu.exe.rsp?
"bin.v2\libs\regex\build\msvc-10.0\debug\threading-multi\has_icu_test.obj" "icuucd.lib" "icudt.lib" "icuind.lib"
comment:8 by , 12 years ago
comment:9 by , 12 years ago
Can you please try again?
There's still an issue of dll-path not being set correctly for 64-bit builds, but otherwise, I *hope* it will now work. Unfortunately I can't test 64-bit support locally though...
Thanks, John.
comment:10 by , 12 years ago
Well, the 32-bits detect ICU and build fine reporting it finished building all targets, however 8 of the 18 resulting files that are built for 1.45 wasn't built.
boost_regex-vc100-mt-gd.lib boost-regex-vc100-mt.lib libboost_regex-vc100-mt-gd.lib libboost_regex-vc100-mt-s.lib libboost_regex-vc100-mt-sgd.lib libboost_regex-vc100-mt.lib libboost_regex-vc100-s.lib libboost_regex-vc100-sgd.lib
are missing.
Win64 build didn't work at all. With the build command
bjam --build-type=complete --with-regex -j2 architecture=x86 address-model=64 stage
error: No best alternative for libs/regex/build/icuuc next alternative: required properties: <link>shared <runtime-link>shared matched next alternative: required properties: <address-model>64 <link>shared <runtime-link>shared matched next alternative: required properties: <link>shared <runtime-link>shared <toolset>msvc <variant>debug matched error: No best alternative for libs/regex/build/icudt next alternative: required properties: <link>shared <runtime-link>shared matched next alternative: required properties: <address-model>64 <link>shared <runtime-link>shared matched next alternative: required properties: <link>shared <runtime-link>shared <toolset>msvc matched error: No best alternative for libs/regex/build/icuin next alternative: required properties: <link>shared <runtime-link>shared matched next alternative: required properties: <address-model>64 <link>shared <runtime-link>shared matched next alternative: required properties: <link>shared <runtime-link>shared <toolset>msvc <variant>debug matched error: No best alternative for libs/regex/build/icudt next alternative: required properties: <link>shared <runtime-link>shared matched next alternative: required properties: <address-model>64 <link>shared <runtime-link>shared matched next alternative: required properties: <link>shared <runtime-link>shared <toolset>msvc matched error: No best alternative for libs/regex/build/icuin next alternative: required properties: <link>shared <runtime-link>shared matched next alternative: required properties: <address-model>64 <link>shared <runtime-link>shared matched next alternative: required properties: <link>shared <runtime-link>shared <toolset>msvc <variant>debug matched error: No best alternative for libs/regex/build/icuuc next alternative: required properties: <link>shared <runtime-link>shared matched next alternative: required properties: <address-model>64 <link>shared <runtime-link>shared matched next alternative: required properties: <link>shared <runtime-link>shared <toolset>msvc <variant>debug matched ............... has_icu_test.obj : error LNK2019: unresolved external symbol u_charFromName_44 referenced in function main bin.v2\libs\regex\build\msvc-10.0\debug\address-model-64\architecture-x86\threading-multi\has_icu.exe : fatal error LNK1120: 1 unresolved externals call "D:\Program Files (x86)\Microsoft Visual Studio 10.0\VC\vcvarsall.bat" x86_amd64 >nul link /NOLOGO /INCREMENTAL:NO /DEBUG /MACHINE:X64 /subsystem:console /out:"bin.v2\libs\regex\build\msvc-10.0\debug\address-model-64\architecture-x86\threading-multi\has_icu.exe" /delayload:icudt44.dll /delayload:icuin44.dll /delayload:icuin44d.dll /delayload:icuuc44.dll /delayload:icuuc44d.dll delayimp.lib @"bin.v2\libs\regex\build\msvc-10.0\debug\address-model-64\architecture-x86\threading-multi\has_icu.exe.rsp" if %ERRORLEVEL% NEQ 0 EXIT %ERRORLEVEL% ...failed msvc.link bin.v2\libs\regex\build\msvc-10.0\debug\address-model-64\architecture-x86\threading-multi\has_icu.exe bin.v2\libs\regex\build\msvc-10.0\debug\address-model-64\architecture-x86\threading-multi\has_icu.pdb... ...removing bin.v2\libs\regex\build\msvc-10.0\debug\address-model-64\architecture-x86\threading-multi\has_icu.pdb ...failed updating 1 target... ...updated 10 targets... ...failed updating 1 target...
comment:11 by , 12 years ago
Aaaah.... in cyberspace no one can hear you scream.... :-0
I had this working - or at least the bjam bit worked - and then I tidied up the Jamfile formatting before committing - and now I just can't get it working again :(
Re the missing files - this is an inevitable consequence of using ICU - it has to link against the dynamic runtime, so the static runtime variants aren't built.
Time to ask for help...
comment:12 by , 12 years ago
follow-up: 15 comment:13 by , 12 years ago
Dave, Yet another attempt to fix this has gone into Trunk... if you haven't lost patience yet, can you give this one a try too?
Cheers, John.
comment:14 by , 12 years ago
Um, I'm not Dave (*point to by line*) but sure. :D
This seems to work. Build without errors for both x86 & x64. All regression tests passed without errors for both as well. :)
KTC
comment:15 by , 12 years ago
Replying to anonymous:
Dave, Yet another attempt to fix this has gone into Trunk... if you haven't lost patience yet, can you give this one a try too?
John, I'll check this out the next time I build an installer, which should be RSN™
comment:16 by , 12 years ago
We've had similar struggles trying to build Boost 1.44 with ICU on 64-bit Windows 7. In addition to the changes to boost/libs/regex/build/Jamfile.v2 which we lifted from Trunk, we also found a fundamental problem with the implementation of check-target-builds in Boost.Build.
The problem is that it does not pass properties such as variant, architecture and address-model from the property set of the "main" build into that for the test executable (has_icu in the case of Boost.Regex). This resulted in attempts to build a debug-mode, 32-bit has_icu in our release-mode 64-bit environment--none of which succeeded.
After a lot of hacking around, we finally made the changes in the attached patch to boost/tools/build/v2/build/configure.jam. I'm not sure these are entirely correct, but when combined with the Boost.Regex Jamfile.v2 from Trunk we were able to successfully build with ICU in both 32- and 64-bit Windows environments.
comment:17 by , 12 years ago
Cc: | added |
---|
by , 12 years ago
Attachment: | check-target-builds-patch added |
---|
Patch for check-target-builds to pass relevant properties along to the target executable
comment:18 by , 12 years ago
Cc: | added |
---|
This seems to be mostly fixed in Trunk anyway - however not all properties are passed down - address-model seems to work OK, but variant debug/release isn't - so I'm adding Vladimir Prus to the CC list.
comment:19 by , 12 years ago
Cc: | added; removed |
---|
comment:20 by , 12 years ago
Shouldn't what's there already which appears to be an improvement on previous release be merged to release for 1.46?
comment:21 by , 12 years ago
Shouldn't what's there already which appears to be an improvement on previous release be merged to release for 1.46?
It has been, sorry forgot to reference the ticket in the commit.
Regards, John.
comment:22 by , 12 years ago
Component: | regex → build |
---|---|
Owner: | changed from | to
Reassigning to Boost.Build as this is now solely an issue with the check-target-builds rule.
Please refer to the comment by cnewbold above to understand the issue.
comment:23 by , 12 years ago
It seems to be that check-target-builds should actually access the list of relevant properties. In some cases, debug vs. release does not actually matter, in particularly on Linux. WDYT?
comment:24 by , 12 years ago
How do you determine what properties are relevant, unless you pass all of them? Not a rhetorical question BTW :-)
comment:25 by , 12 years ago
The plan is for me not to answer the above non-rhetorical question, by asking the user of check-target-builds to explicitly specify relevant properties if those are different from the default set (which have to documented, but that's separate matter).
comment:26 by , 12 years ago
I don't see how that is a solution - how can the author of the Jamfile know what build properties may be specified on the command line - for example I don't see how this addresses the original issue above - that variant, address-model and architechture options don't get passed to the config check which causes incorrect configuration.
To take another example - what happens if the user is building both 32 and 64 bit binaries, and the configuration for each is different?
comment:27 by , 12 years ago
First, not that address-model and architecture are now passed.
Second, it's not a random Jamfile author -- it's somebody who implements a configuration check, and while that check is in Jamfile in this specific case, he might as well be in a special module. Given that you're implementing a configuration check, you should know what properties determine its outcome. In case of ICU, you should know whether debug or release variant determines the outcome. The only other alternative is to re-run a check for every possible set of properties ever used anywhere, which might waste time.
comment:28 by , 12 years ago
Well OK, pretty much all the properties may have an impact - certainly include, search, cxxflags, linkflags, also release and debug (on windows because both ICU lib variants may not be built, and only release mode versions are in the binary ICU distro), probably also define on MSVC (because some defines change the std lib ABI), and I'm sure others I haven't thought of yet... And that's really the problem, on some given platform, I really don't know the complete list of - possibly-compiler-specific - options that will have an effect.
For that matter, how on earth do I force a check-target-builds rule to use the same properties as the target that's using it anyway? Remember also that the target may be built more than once with different property sets - and therefore need different configurations.
Cheers, John.
comment:29 by , 12 years ago
Cc: | added |
---|
I just discovered this ticket. Using Microsoft Visual C++ 10.0, I was able to build Boost.Regex (Boost 1.46.1) with ICU 4.6.1 support by modifying the has_icu.exe.rsp
file and re-running the link
command.
The contents of the has_icu.exe.rsp
file were originally:
"bin.v2\libs\regex\build\msvc-10.0\debug\address-model-64\architecture-x86\threading-multi\has_icu_test.obj" "icudt.lib" "icuind.lib" "icuucd.lib"
Notice the trailing "d" on the last two library names. By removing those two "d"s and re-running the link
command, I was able to build has_icu.exe
.
follow-up: 31 comment:30 by , 12 years ago
Notice the trailing "d" on the last two library names. By removing those two "d"s and >re-running the link command, I was able to build has_icu.exe.
This isn't really a 64-bit issue - it's a debug build vs release build issue.
The ICU dll's built against the release MSVC runtime, don't have the "d" suffix, the debug ones do (but you have to build them yourself from the project files). Note that while you can link the release ICU versions to a debug build, doing so will cause two different runtime libraries to be in use, and that almost always results in a crash or heap curruption at some point :(
To put it another way - the build script is correct - when building debug regex libraries it looks for the debug ICU dll's because those are the only ones that are guaranteed to work correctly. It's just that ICU doesn't supply those in it's binary release .zips, so you have to build them yourself.
HTH, John.
comment:31 by , 12 years ago
This isn't really a 64-bit issue - it's a debug build vs release build issue.
The ICU dll's built against the release MSVC runtime, don't have the "d" suffix, the debug ones do (but you have to build them yourself from the project files). Note that while you can link the release ICU versions to a debug build, doing so will cause two different runtime libraries to be in use, and that almost always results in a crash or heap curruption at some point :(
To put it another way - the build script is correct - when building debug regex libraries it looks for the debug ICU dll's because those are the only ones that are guaranteed to work correctly. It's just that ICU doesn't supply those in it's binary release .zips, so you have to build them yourself.
Oh, I know that the "d" means "debug". The problem is that I am not building debug Boost.Regex; I am only building the release variant (in the bjam
line I included variant=release
).
comment:33 by , 12 years ago
Unfortunately I have a complication to add to the subject.
I compiled ICU 4.6.1 on AIX 6.1 using the following line:
./configure --prefix=/some/icupath --enable-rpath --disable-debug --disable-tracing --with-library-bits=64
The latter was necessary because the default behaviour of ICU's configure on AIX is to build 32bit binaries even while the OS is 64bit.
The resulting libraries and binaries reside in the subdirectories /some/icupath/bin and /some/icupath/lib (no "64" in the paths).
When I try to compile Boost 1.46.1, like so:
./bootstrap.sh --prefix=/some/path --with-toolset=vacpp --with-icu=/some/icupath --with-libraries=regex ./bjam address-model=64
Boost concludes that:
vacpp.compile.c++ bin.v2/libs/regex/build/vacpp/debug/address-model-64/has_icu_test.o vacpp.link bin.v2/libs/regex/build/vacpp/debug/address-model-64/has_icu ld: 0711-317 ERROR: Undefined symbol: .u_charFromName_46 ld: 0711-345 Use the -bloadmap or -bnoquiet option to obtain more information. xlC -brtl -bnoipath -q64 -g -qfullpath -o "bin.v2/libs/regex/build/vacpp/debug/address-model-64/has_icu" -L/PAZ/contrib_test/icu-4.6.1/bin64 -L/PAZ/contrib_test/icu-4.6.1/lib64 "bin.v2/libs/regex/build/vacpp/debug/address-model-64/has_icu_test.o" -licuuc -licui18n -licudata
Long story short, ICU uses lib and bin, while Boost's ICU tests looks into lib64 and bin64...
comment:34 by , 10 years ago
John,
Do you need access to a Win64 system where you can test this?
P.S. If you don't want to sign in, just enter your username (johnmaddock) in the Author ticket at the bottom when you comment
follow-up: 36 comment:35 by , 10 years ago
Do you need access to a Win64 system where you can test this?
I think it's moved far beyond Win64 now - actually I think those issues are mostly solved? But I assume we still have problems with Boost.Build not passing all the build variants on to the config check.
The resulting libraries and binaries reside in the subdirectories /some/icupath/bin and /some/icupath/lib (no "64" in the paths)
I think we may have reached the point at which we can't please everybody all of the time :-(
For a specific setup like that, I'd suggest building with:
export LD_LIBRARY_PATH=some-path/bin bjam include=/some-path/include linkflags=-Lsome-path/lib
So everything is "just there" in the compilers paths.
comment:36 by , 10 years ago
Replying to johnmaddock:
Do you need access to a Win64 system where you can test this?
I think it's moved far beyond Win64 now - actually I think those issues are mostly solved?
I don't know how to tell whether those issues are solved or not.
But I assume we still have problems with Boost.Build not passing all the build variants on to the config check.
I don't know how to tell that either.
The resulting libraries and binaries reside in the subdirectories /some/icupath/bin and /some/icupath/lib (no "64" in the paths)
I think we may have reached the point at which we can't please everybody all of the time :-(
Personally I don't care about the path to the resulting libraries. I'm looking to create a 64-bit build with ICU on Windows.
comment:37 by , 10 years ago
Personally I don't care about the path to the resulting libraries. I'm looking to create a 64-bit build with ICU on Windows.
I believe that part should have been fixed - do you still see issues?
John.
comment:38 by , 10 years ago
Yes, I still see issues. I don't remember the exact issues but there were problems when I tried to do this recently or I wouldn't have bumped this ticket. I'll try to find some time to repeat the experiment and write down the specifics.
Note that even after accounting for that issue, ICU is still not successfully detected under x64: