Boost C++ Libraries: Ticket #9766: boost >= 1.54 failes to compile with gcc-4.8.2 and LTO enabled

attachment set

steffen@… — Mon, 10 Mar 2014 13:01:23 GMT

attachment → boost-1.55_build.log.bz2

boost 1.55 build log

steffen@… — Wed, 23 Apr 2014 10:47:27 GMT

No one ever looked at this issue?

component changed; owner set

viboes — Wed, 23 Apr 2014 10:59:48 GMT

owner set to Andrey Semashev
component Building Boost → log

Try moving it to Log library

Andrey Semashev — Wed, 23 Apr 2014 11:46:42 GMT

Can you provide the b2 or bjam command line so I could reproduce the error? BTW, I compile with gcc 4.8.2 on Kubuntu without problems.

anonymous — Wed, 23 Apr 2014 12:22:58 GMT

Hi,

this is the command, Gentoo's ebuild used to build boost:

b2 gentoorelease -j5 -q -d+2 --user-config=/home/misc/gentoo/tmp/portage/dev-libs/boost-1.55.0-r1/work/boost_1_55_0/user-config.jam -sICU_PATH=/usr --without-mpi --without-python --without-context --without-coroutine pch=off --boost-build=/usr/share/boost-build --prefix="/home/misc/gentoo/tmp/portage/dev-libs/boost-1.55.0-r1/image/usr" --layout=system threading=multi link=shared

and this is the content of user-config.jam:

using gcc : 4.8 : x86_64-pc-linux-gnu-g++ : <cflags>"-march=native -O2 -pipe -ggdb -floop-interchange -floop-strip-mine -floop-block -ftree-loop-distribution -fira-loop-pressure -ftree-vectorize -ftree-loop-linear -flto=5 -fuse-linker-plugin" <cxxflags>"-march=native -O2 -pipe -ggdb -floop-interchange -floop-strip-mine -floop-block -ftree-loop-distribution -fira-loop-pressure -ftree-vectorize -ftree-loop-linear -flto=5 -fuse-linker-plugin -std=gnu++98" <linkflags>"-Wl,-O1 -Wl,--as-needed -march=native -O2 -pipe -ggdb -floop-interchange -floop-strip-mine -floop-block -ftree-loop-distribution -fira-loop-pressure -ftree-vectorize -ftree-loop-linear -flto=5 -fuse-linker-plugin -Wl,-znow -Wl,--sort-common -Wl,--hash-style=gnu -Wl,--enable-new-dtags" ;

Please let me know if you need further information.

anonymous — Wed, 23 Apr 2014 12:41:10 GMT

I can see you're using LTO. That may be the source of the problem. There are several source files in Boost.Log which are compiled with different compiler flags. In particular, it's dump_avx2.cpp, which contains the failing function you seen in the compiler errors. To verify that try disabling LTO and rebuilding the library.

I have no experience with GCC LTO, but from what I know I can't use all the flags on the linking stage since that would make the whole resulting binary AVX2-only instead of only just the optimized part in dump_avx2.cpp. I could try adding attributes to the particular functions in dump_avx2.cpp so that they contain the required compiler options. But really, this has to be done by the compiler since I already pass the options on the compilation stage.

anonymous — Wed, 23 Apr 2014 13:09:34 GMT

Have you been able to reproduce the failure with my flags? Disabling LTO fixes the issue, sorry I forgot to mention that in my initial bug report. I guess this bug should be reported upstream to the GCC devs? Can you do this or do I have to deal with it? I'm wondering why I haven't found any other guys around who fail to compile boost >=1.54 with LTO enabled.

Andrey Semashev — Wed, 23 Apr 2014 18:33:43 GMT

Replying to anonymous:

Have you been able to reproduce the failure with my flags?

No, I have tried to compile git develop branch with your flags and it succeeded without errors. As I said, I have gcc 4.8.2 on Ubuntu. I'm not sure how to verify that LTO was actually used. The object files (*.o) have the __gnu_lto_v1 symbol, if that's an indication. OTOH, objdump produces disassembly for these files, which I find surprising.

Disabling LTO fixes the issue, sorry I forgot to mention that in my initial bug report. I guess this bug should be reported upstream to the GCC devs? Can you do this or do I have to deal with it? I'm wondering why I haven't found any other guys around who fail to compile boost >=1.54 with LTO enabled.

I suppose, LTO is not used very often yet in Linux world. Since I cannot reproduce the failure, it's probably easier for you to communicate with gcc devs (or package maintainers in Gentoo) regarding this issue.

steffen@… — Fri, 25 Apr 2014 12:29:25 GMT

I tried to compile it under OpenSUSE 13.1 (gcc 4.8.1) on another system (also Core i7 Sandy Bridge) and it failed there with the same error.

Then I tried to compile it on my older computer under Ubuntu 14.04 (AMd Athlon X2 4600), and it failed with an internal compiler error, I guess due to the lack of AVX support in that old CPU:

libs/log/src/dump_avx2.cpp: In function ‘_ZN5boost3log11v2_mt_posix3aux20dump_data_wchar_avx2EPKvmRSt13basic_ostreamIwSt11char_traitsIwEE.part.4’:
libs/log/src/dump_avx2.cpp:260:6: warning: AVX vector return without AVX enabled changes the ABI [-Wpsabi]
 void dump_data_wchar_avx2(const void* data, std::size_t size, std::basic_ostream< wchar_t >& strm)
      ^
libs/log/src/dump_avx2.cpp:260:6: internal compiler error: in convert_move, at expr.c:316
Please submit a full bug report,
with preprocessed source if appropriate.
See <file:///usr/share/doc/gcc-4.8/README.Bugs> for instructions.

Did you use my user-config.jam? Then bjam should have called g++ with "-flto=5" option. You can see the compiler options in the output lines of bjam. I'll file a bug report for this to GCC and give you the id.

Andrey Semashev — Fri, 25 Apr 2014 12:45:33 GMT

Replying to steffen@…:

Then I tried to compile it on my older computer under Ubuntu 14.04 (AMd Athlon X2 4600), and it failed with an internal compiler error, I guess due to the lack of AVX support in that old CPU:

Host CPU features should not influence the code generation, unless you use -march=native. BTW, you shouldn't really use -march and similar flags, b2 has instruction-set property, which has the same meaning but may also be picked up by Boost libraries build scripts.

Did you use my user-config.jam?

Yes, I only had to change the compiler executable name and correct directories and similar stuff in the command line. I'll try experimenting more though.

Then bjam should have called g++ with "-flto=5" option. You can see the compiler options in the output lines of bjam.

Yes, all options were there, both at compilation and linking stages. I wonder if it could be related to -march=native flag since I tried this on a Haswell machine. I'll try on another machine when I have time.

I'll file a bug report for this to GCC and give you the id.

Thanks, I appreciate that.

Andrey Semashev — Sat, 26 Apr 2014 17:35:52 GMT

After all my experiments I still fail to reproduce the error. I even tried to replay the compilation with the command lines from your build log. I've discovered one nasty detail though - the resulting binaries in my case are AVX2-only. I guess my compiler somehow applies the flags for dump_avx2.cpp to all object files. Needless to say that if that's how LTO is supposed to work, it cannot be used with projects such as Boost.Log.

However, I have a patch I'd like to test. Basically, it adds attributes to all SSE/AVX functions so that the compiler knows what CPU they are targeted for. I'd appreciate if you could perform the following experiment:

Apply the patch from this ticket.
Perform a clean rebuild of Boost.
If Boost.Log compiles this time, run the following command:

objdump -dS libboost_log.so.1.55.0 >libboost_log.so.1.55.0.S

(add the directory name to the compiled library, if necessary).

Compress libboost_log.so.1.55.0.S and attach it to this ticket.

The libboost_log.so.1.55.0.S is the disassembled library; I'd like to inspect the resulting code for whether it's AVX2-only or not.

attachment set

Andrey Semashev — Sat, 26 Apr 2014 17:36:41 GMT

attachment → add_target_attribs.patch

The patch adds target CPU attributes to the optimized functions.

steffen@… — Tue, 29 Apr 2014 16:24:38 GMT

I have reported it upstream. Here is the link: gcc.gnu.org/bugzilla/show_bug.cgi?id=60964. Seems like your suggestion was really close with -march=native (I used this as it was suggested in wiki.gentoo.org/wiki/CFLAGS). They asked for a test case. May you provide one based on your log library?

I also tried to compile with -march=corei7-avx and that solved the issue for me. I may also try to compile with your patch.

Andrey Semashev — Tue, 29 Apr 2014 18:33:48 GMT

May you provide one based on your log library?

I'm still not able to reproduce the failure, sorry.

anonymous — Wed, 30 Apr 2014 10:16:36 GMT

Could you please give some detail about your used platform? Which Ubuntu release/GCC version are you using? This bug is only present in GCC 4.8 and newer. I've now checked also Fedora 20 and had the same issue as under Gentoo and current OpenSUSE.

I'll try to compile boost with your patch and -march=native next week as I'm on business trip until next week.

Andrey Semashev — Wed, 30 Apr 2014 10:20:54 GMT

I'm on Kubuntu 13.10 x86_64, gcc -v gives:

gcc -v
Using built-in specs.
COLLECT_GCC=/usr/bin/gcc-4.8.real
COLLECT_LTO_WRAPPER=/usr/lib/gcc/x86_64-linux-gnu/4.8/lto-wrapper
Target: x86_64-linux-gnu
Configured with: ../src/configure -v --with-pkgversion='Ubuntu/Linaro 4.8.1-10ubuntu9' --with-bugurl=file:///usr/share/doc/gcc-4.8/README.Bugs --enable-languages=c,c++,java,go,d,fortran,objc,obj-c++ --prefix=/usr --program-suffix=-4.8 --enable-shared --enable-linker-build-id --libexecdir=/usr/lib --without-included-gettext --enable-threads=posix --with-gxx-include-dir=/usr/include/c++/4.8 --libdir=/usr/lib --enable-nls --with-sysroot=/ --enable-clocale=gnu --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-gnu-unique-object --enable-plugin --with-system-zlib --disable-browser-plugin --enable-java-awt=gtk --enable-gtk-cairo --with-java-home=/usr/lib/jvm/java-1.5.0-gcj-4.8-amd64/jre --enable-java-home --with-jvm-root-dir=/usr/lib/jvm/java-1.5.0-gcj-4.8-amd64 --with-jvm-jar-dir=/usr/lib/jvm-exports/java-1.5.0-gcj-4.8-amd64 --with-arch-directory=amd64 --with-ecj-jar=/usr/share/java/eclipse-ecj.jar --enable-objc-gc --enable-multiarch --disable-werror --with-arch-32=i686 --with-abi=m64 --with-multilib-list=m32,m64,mx32 --with-tune=generic --enable-checking=release --build=x86_64-linux-gnu --host=x86_64-linux-gnu --target=x86_64-linux-gnu
Thread model: posix
gcc version 4.8.1 (Ubuntu/Linaro 4.8.1-10ubuntu9)

It's the standard compiler installed from the Ubuntu packages.

status changed; resolution set

Andrey Semashev — Sat, 03 May 2014 10:05:07 GMT

status new → closed
resolution → fixed

I managed to reproduce the error, after I upgraded to Kubuntu 14.04 and tried to compile Boost with -march=native on a Sandy Bridge machine. The patch I attached earlier does not help. For now I have no further ideas how to work it around and make -march=native work, the issue has to be fixed in gcc.

Considering that even when the compilation succeeds (i.e. -march=native is not used) the resulting binaries are incorrect, I'll document gcc LTO as unsupported.

Andrey Semashev — Sat, 03 May 2014 12:31:19 GMT

BTW, I reported another LTO bug: http://gcc.gnu.org/bugzilla/show_bug.cgi?id=61043

steve@… — Sat, 13 Feb 2016 23:19:43 GMT

This isn't a GCC bug. Use of -march is expected to produce code suitable for *only* the target CPU architecture. Usage of -march or other -m flags to enable features from within build-systems is incorrect usage of the compiler. The build-system should detect which features are available on the configured compiler, not the other way around. Generic code should be generated with with a specific minimum requirement in mind, but should always be limited to whatever the build toolchain is constrained by.

Andrey Semashev — Sat, 13 Feb 2016 23:25:07 GMT

Read the discussion and the referred bug. The architecture-specific flags are passed for compilation of the files, which are executed only when the relevant CPU features are present in run time. The compiler bug is that it produces binary code as if the flags were specified for all files rather than those few special ones.

steve@… — Sun, 14 Feb 2016 10:04:22 GMT

My point is though that even when generating generic code with run-time detection it's inappropriate to override -march. If generic code is required -march can be used to specify a generic target (such as x86_64, or i686) and then use feature -m switches. If building not building generic code, it should generally be constrained to the specific set of enabled features, and certainly shouldn't override the per target code generation optimization. The GCC bug recommends using attribute(target("...")) which allows specific functions to use features not necessarily available for the given -march, and which doesn't fail for LTO, and actually produces the desired behaviour. See: https://gcc.gnu.org/onlinedocs/gcc/Function-Specific-Option-Pragmas.html

Andrey Semashev — Sun, 14 Feb 2016 10:21:33 GMT

Specifying different compiler options for different translation units is a common practice, I'm not sure what you're arguing for.

The bug report also mentions that the attributes don't work up until gcc 4.9. And the bug is fixed proper in 5.1. I do not think mangling the code just for 4.9 is worth the effort.

anonymous — Sun, 14 Feb 2016 11:10:59 GMT

Actually, I'm using gcc-5.3, and it still causes build failure on amdfam10 (AMD Phenom II). I'm arguing that overriding -march isn't one of the compiler options that should be set for different translation units. This is something that often crops up in Gentoo, where -march is set system-wide, and then gets overridden by upstream build-systems not following best practises. There are plenty of patches in Portage to work around these issues.

The simplest solution is to just make the optimized functions individually build-time configurable, that way a generic distributor build can chose a suitable generic -march, along with enabling run-time CPU optimizations. Conversely, a targeted build can just include support and optimization as appropriate.

I brought up the function attributes because that's the solution GCC has implemented to support this type of situation as was discussed in the GCC bug report, but as you say, it's only good for GCC>4.9.

Andrey Semashev — Sun, 14 Feb 2016 18:52:43 GMT

Actually, I'm using gcc-5.3, and it still causes build failure on amdfam10 (AMD Phenom II).

Could you provide more details on the failure? Is the error reproducible with the latest vanilla Boost release?

The simplest solution is to just make the optimized functions individually build-time configurable

That is too difficult to maintain.