Boost C++ Libraries: Ticket #9743: Patch for context/coroutine assembler for Linux On Power (BE & LE)

attachment set

Wed, 05 Mar 2014 10:01:26 GMT

attachment → jump_ppc64_sysv_elf_gas.S

attachment set

Wed, 05 Mar 2014 10:01:43 GMT

attachment → make_ppc64_sysv_elf_gas.S

olli — Wed, 05 Mar 2014 10:23:48 GMT

Hello Toni, I suggest to combine little and big endian into one file for make_fcontext and jump_fcontext. Could you adapt your patches according to the code of the develop branch? In this new version the registers will be stored at the current end of the stack.

Wed, 05 Mar 2014 12:58:30 GMT

Replying to olli:

Hello Toni, I suggest to combine little and big endian into one file for make_fcontext and jump_fcontext. Could you adapt your patches according to the code of the develop branch?

Hello Olli, Hummm You seem to say that changes have occured about this subject between 1.55 and now. So, yes, I have to look at develop branch.

Moreover, instead of using the LITTLE_ENDIAN and BIG_ENDIAN, I should use something like:

#if !defined POWERPC64

_CALL_ELF == 2

In this new version the registers will be stored at the current end of the stack.

OK. I have to manage these changes.

Thanks

Wed, 05 Mar 2014 15:14:51 GMT

Looking at build/Jamfile.v2 file, I see changes. Probably also within .S files. I'm now rebuilding my Boost environment on LoP BE & LE with this develop version. So that I can compare and test, and build a new patch, I guess. Tony

anonymous — Wed, 05 Mar 2014 16:18:08 GMT

please note that you have to remove the untested.cpp file from build/Jamfile.v2 for the PPC target

Wed, 05 Mar 2014 16:32:41 GMT

I have removed the untested.cpp file for POWER64 from context/build/Jamfile.vé . However, I still have the following error while trying to run the tests of context :

[root@tbx6 build]# cd - /home/reixt/BOOST/modular-boost/status [root@tbx6 status]# ../b2 Performing configuration checks

symlinks supported : yes (cached)
hardlinks supported : yes (cached)
32-bit : no (cached)
64-bit : yes (cached)
arm : no (cached)
mips1 : no (cached)
power : yes (cached)
lockfree boost::atomic_flag : yes (cached)
has_icu builds : yes (cached)

...patience... ...patience... ...found 4421 targets... ...updating 13 targets... Jamfile</home/reixt/BOOST/modular-boost/libs/context/build>.gas ../bin.v2/libs/context/build/gcc-4.8.2/debug/address-model-64/architecture-power/link-static/asm/make_ppc64_sysv_elf_gas.o ../libs/context/src/asm/make_ppc64_sysv_elf_gas.S: Messages de l'assembleur: ../libs/context/src/asm/make_ppc64_sysv_elf_gas.S:98: Erreur: pseudo-op inconnu: « .localentry » ../libs/context/src/asm/make_ppc64_sysv_elf_gas.S:102: Erreur: symbole « make_fcontext » est déjà défini

as -o "../bin.v2/libs/context/build/gcc-4.8.2/debug/address-model-64/architecture-power/link-static/asm/make_ppc64_sysv_elf_gas.o" "../libs/context/src/asm/make_ppc64_sysv_elf_gas.S"

...failed Jamfile</home/reixt/BOOST/modular-boost/libs/context/build>.gas ../bin.v2/libs/context/build/gcc-4.8.2/debug/address-model-64/architecture-power/link-static/asm/make_ppc64_sysv_elf_gas.o... Jamfile</home/reixt/BOOST/modular-boost/libs/context/build>.gas ../bin.v2/libs/context/build/gcc-4.8.2/debug/address-model-64/architecture-power/link-static/asm/jump_ppc64_sysv_elf_gas.o ../libs/context/src/asm/jump_ppc64_sysv_elf_gas.S: Messages de l'assembleur: ../libs/context/src/asm/jump_ppc64_sysv_elf_gas.S:98: Erreur: pseudo-op inconnu: « .localentry » ../libs/context/src/asm/jump_ppc64_sysv_elf_gas.S:102: Erreur: symbole « jump_fcontext » est déjà défini

as -o "../bin.v2/libs/context/build/gcc-4.8.2/debug/address-model-64/architecture-power/link-static/asm/jump_ppc64_sysv_elf_gas.o" "../libs/context/src/asm/jump_ppc64_sysv_elf_gas.S"

...failed Jamfile</home/reixt/BOOST/modular-boost/libs/context/build>.gas ../bin.v2/libs/context/build/gcc-4.8.2/debug/address-model-64/architecture-power/link-static/asm/jump_ppc64_sysv_elf_gas.o... ...skipped <p../bin.v2/libs/context/build/gcc-4.8.2/debug/link-static>libboost_context.a(clean) for lack of <p../bin.v2/libs/context/build/gcc-4.8.2/debug/address-model-64/architecture-power/link-static>asm/make_ppc64_sysv_elf_gas.o... ...skipped <p../bin.v2/libs/context/build/gcc-4.8.2/debug/link-static>libboost_context.a for lack of <p../bin.v2/libs/context/build/gcc-4.8.2/debug/address-model-64/architecture-power/link-static>asm/make_ppc64_sysv_elf_gas.o... ...skipped <p../bin.v2/libs/context/test/test_context.test/gcc-4.8.2/debug/link-static>test_context for lack of <p../bin.v2/libs/context/build/gcc-4.8.2/debug/link-static>libboost_context.a... ...skipped <p../bin.v2/libs/context/test/test_context.test/gcc-4.8.2/debug/link-static>test_context.run for lack of <p../bin.v2/libs/context/test/test_context.test/gcc-4.8.2/debug/link-static>test_context... ...skipped <p../bin.v2/libs/coroutine/test/test_asymmetric_coroutine.test/gcc-4.8.2/debug/link-static>test_asymmetric_coroutine for lack of <p../bin.v2/libs/context/build/gcc-4.8.2/debug/link-static>libboost_context.a... ...skipped <p../bin.v2/libs/coroutine/test/test_asymmetric_coroutine.test/gcc-4.8.2/debug/link-static>test_asymmetric_coroutine.run for lack of <p../bin.v2/libs/coroutine/test/test_asymmetric_coroutine.test/gcc-4.8.2/debug/link-static>test_asymmetric_coroutine... ...skipped <p../bin.v2/libs/coroutine/test/test_symmetric_coroutine.test/gcc-4.8.2/debug/link-static>test_symmetric_coroutine for lack of <p../bin.v2/libs/context/build/gcc-4.8.2/debug/link-static>libboost_context.a... ...skipped <p../bin.v2/libs/coroutine/test/test_symmetric_coroutine.test/gcc-4.8.2/debug/link-static>test_symmetric_coroutine.run for lack of <p../bin.v2/libs/coroutine/test/test_symmetric_coroutine.test/gcc-4.8.2/debug/link-static>test_symmetric_coroutine... ...failed updating 2 targets... ...skipped 11 targets...

anonymous — Wed, 05 Mar 2014 18:28:18 GMT

try to add following to build/Jamfile.v2:

alias asm_context_sources

: asm/make_ppc64_sysv_elf_gas.S

asm/jump_ppc64_sysv_elf_gas.S

: <address-model>64

<architecture>power <binary-format>elf <toolset>gcc

;

Thu, 06 Mar 2014 10:07:36 GMT

Hi anonymous,

There are several errors in the current Boost code for PPC64. (tests below were done in LoP PPC64/BE environment)

1) src/asm/make_ppc64_sysv_elf_gas.S This file contains use of macros, like: #if _CALL_ELF == 2 However, in build/Jamfile.v2, the gas action is defined as: actions gas {

as -o "$(<)" "$(>)"

} and that does not expend the Macros. There is a need of calling cpp.

2) Adding the lines you gave me in previous comment still leads to an error (both on BE and LE): [root@tbx6 build]# pwd /home/reixt/BOOST/modular-boost/libs/context/build [root@tbx6 build]# bjam warning: mismatched versions of Boost.Build engine and core warning: Boost.Build engine (bjam) is 2011.12.01 warning: Boost.Build core (at /home/reixt/BOOST/modular-boost/tools/build/src) is 2013.05-svn Performing configuration checks

symlinks supported : yes
hardlinks supported : yes

/home/reixt/BOOST/modular-boost/tools/build/src/tools/link.jam:314: in link-recursively ERROR: rule "READLINK" unknown in module "link". ........

Maybe I have some mis-match in my environment. Or I'm not building in the correct place ?

3) src/asm/make_ppc64_sysv_elf_gas.S contains PPC 32bits assembler instructions, and it mixes PPC 32bits conventions with 64bits. So it cannot work.

I need some clarifications.

Thanks

Tony

olli — Thu, 06 Mar 2014 12:47:58 GMT

1) you could try the modified Jamfile -> the assembler files are passed to the preprocessor before applied to gas.

2) I guess it's a problem of the boost.build system

3) could you identify which instructions must be corrected and generate a patch for it?

attachment set

olli — Thu, 06 Mar 2014 12:48:35 GMT

attachment → Jamfile.v2

preprocessor before gas

Thu, 06 Mar 2014 14:32:46 GMT

Hi,

I'm nuts about complex assembler stuff. However I'm helped by an expert. He has found the following:

1) First, it seems that the API of jump_fcontext() has been changed (from 1.55 to today).

intptr_t jump_fcontext( fcontext_t * ofc, fcontext_t const* nfc, intptr_t vp, bool preserve_fpu = true);

intptr_t jump_fcontext( fcontext_t * ofc, fcontext_t nfc, intptr_t vp, bool preserve_fpu = true);

2) It seems that you have made changes for Power that are not valid for Linux on Power (LoP), both BE and LE.

So, we propose the following changes (see files that I gonna attach to the ticket) that are valid to Linux on PowerPC Big Endian or Little Endian (there is only 64bits systems, no 32bits systems):

make_fcontext :
- use of 64bits opcodes (std instead of stw)
- about ELFv1 case: we grab the address and TOC of the context function.

jump_fcontext :
- about ELFv1 case: we save/restore the TOC.
- we don't save/restore R13
- about ELFv2: the address of the context function is passed through R12

4) We use the Macros:

_CALL_ELF == 2 _CALL_LINUX

3) These jump/make files must be preprocessed by cpp, in order to process the Macros. I'm not sure that it works now with Jamfile.v2. My colleague tested his code in a much simpler environment. I gonna check asap.

We have tested the files I gonna attach both on LoP BE and LE. Only 64bits.

jump_ppc64_sysv_elf_gas.S make_ppc64_sysv_elf_gas.S

In order to prevent confusion, I suggest that either you rename these 2 files another way (jump/make_ppc64_linux_elf_gas.S) or you mix all versions together (which may be difficult to maintain). I would suggest first solution. However, I do not yet master what the Jamfile.v2 of context lib does.

Regards,

Tony

attachment set

Thu, 06 Mar 2014 14:34:56 GMT

attachment → jump_ppc64_linux_elf_gas.S

Assembler code for Linux on Power BE/LE PPC64

attachment set

Thu, 06 Mar 2014 14:35:14 GMT

attachment → make_ppc64_linux_elf_gas.S

Assembler code for Linux on Power BE/LE PPC64

Thu, 06 Mar 2014 14:36:34 GMT

I've attached the 2 new files, named:

jump_ppc64_linux_elf_gas.S make_ppc64_linux_elf_gas.S

(but use the name that seems appropriate for you).

olli — Thu, 06 Mar 2014 18:12:45 GMT

Hi Tony, thx for the paches. I've changed the Jamfile - the asm files will be passed to the preprocessor before GNU as. Could you pull the last changes from git repo (branch develop) and execute the unit-tests (sub-directory test; call b2/bjam at command line)? best, Oliver

Fri, 07 Mar 2014 10:58:51 GMT

Hi Oliver,

I've pulled the last changes in context directory from "develop" to my Ubuntu PC, and I've copied the files on my tests machines. However, I still have an error when running bjam in "modular-boost/libs/context/build" (on my test machines and on my PC), though it worked fine with v1.55 :

tony@tony2:~/BOOST/modular-boost/libs/context/build$ bjam

symlinks supported : yes
hardlinks supported : yes

/home/tony/BOOST/modular-boost/tools/build/src/tools/link.jam:314: in link-recursively ERROR: rule "READLINK" unknown in module "link".

Any idea about why this does not work ?

Anyway: If I run ./b2 at top and then run ../b2 in status directory, that works fine, and I've been able to check that all the context and coroutine tests are "passed", both on LoP LE (Ubuntu 14.04) and LOP BE (Fedora 19).

Jamfile</home/tony/BOOST/modular-boost/libs/context/build>.gas ../bin.v2/libs/context/build/gcc-4.8/debug/address-model-64/architecture-power/link-static/asm/make_ppc64_sysv_elf_gas.o Jamfile</home/tony/BOOST/modular-boost/libs/context/build>.gas ../bin.v2/libs/context/build/gcc-4.8/debug/address-model-64/architecture-power/link-static/asm/jump_ppc64_sysv_elf_gas.o

So, that seems perfect for Linux On Power in LE and BE modes ! :)

However, please check that it does not break other PPC64 environment.

Regards,

Tony

olli — Fri, 07 Mar 2014 11:42:50 GMT

Replying to Tony Reix <tony.reix@…>:

Hi Oliver,

I've pulled the last changes in context directory from "develop" to my Ubuntu PC, and I've copied the files on my tests machines. However, I still have an error when running bjam in "modular-boost/libs/context/build" (on my test machines and on my PC), though it worked fine with v1.55 :

tony@tony2:~/BOOST/modular-boost/libs/context/build$ bjam

symlinks supported : yes
hardlinks supported : yes
/home/tony/BOOST/modular-boost/tools/build/src/tools/link.jam:314: in link-recursively ERROR: rule "READLINK" unknown in module "link".

Any idea about why this does not work ?

no idea - boost/tools/build/src/tools/link.jam is part of the boost.build system and I'm not an expert of boost.build. maybe you should add an bug for boost.build?

Anyway: If I run ./b2 at top and then run ../b2 in status directory, that works fine, and I've been able to check that all the context and coroutine tests are "passed", both on LoP LE (Ubuntu 14.04) and LOP BE (Fedora 19).

Jamfile</home/tony/BOOST/modular-boost/libs/context/build>.gas ../bin.v2/libs/context/build/gcc-4.8/debug/address-model-64/architecture-power/link-static/asm/make_ppc64_sysv_elf_gas.o Jamfile</home/tony/BOOST/modular-boost/libs/context/build>.gas ../bin.v2/libs/context/build/gcc-4.8/debug/address-model-64/architecture-power/link-static/asm/jump_ppc64_sysv_elf_gas.o

So, that seems perfect for Linux On Power in LE and BE modes ! :)

that's really nice - I'll merge the changes to branch master

However, please check that it does not break other PPC64 environment.

unfortunately I've no PPC64 system and boost does not have PPC64 regression tests for this platform.

best, Oliver

status changed; resolution set

olli — Fri, 07 Mar 2014 16:13:19 GMT

status new → closed
resolution → fixed

Steven Watanabe — Fri, 07 Mar 2014 16:53:59 GMT

Replying to Tony Reix <tony.reix@…>:

tony@tony2:~/BOOST/modular-boost/libs/context/build$ bjam

symlinks supported : yes
hardlinks supported : yes
/home/tony/BOOST/modular-boost/tools/build/src/tools/link.jam:314: in link-recursively ERROR: rule "READLINK" unknown in module "link".

Any idea about why this does not work ?

Your b2 is out of date. Please rebuild it. READLINK is a new builtin rule which is needed to support modular boost.

Tue, 11 Mar 2014 10:05:13 GMT

Replying to steven_watanabe:

Replying to Tony Reix <tony.reix@…>:

tony@tony2:~/BOOST/modular-boost/libs/context/build$ bjam

symlinks supported : yes
hardlinks supported : yes
/home/tony/BOOST/modular-boost/tools/build/src/tools/link.jam:314: in link-recursively ERROR: rule "READLINK" unknown in module "link".

Any idea about why this does not work ?

Your b2 is out of date. Please rebuild it. READLINK is a new builtin rule which is needed to support modular boost.

Hummm AFAIK, rebuilding b2 is done by: 1) remove it, 2) launch again ./bootstrap.sh . I've done this and it produced exactly the same file b2. Is there a special process I'm not aware of for rebuilding b2 ? Thx

ad@… — Wed, 12 Mar 2014 23:27:06 GMT

I have a regression from 1.55 final to 1.56 develop and this change

gcc-4.8.2 => target-os=windows => ok clang-darwin-4.2.1 => target-os=darwin => failure

"cpp" is not known (I may symlink cpp to clang.exe) but after "as" doesn't know the switch -am

ad@… — Wed, 12 Mar 2014 23:29:08 GMT

I have a regression from 1.55 final to 1.56 develop and this change

gcc-4.8.2 => target-os=windows => ok
gcc-4.8.2 => target-os=linux => ok
clang-darwin-4.2.1 => target-os=darwin => failure

"cpp" is not known (I may symlink cpp to clang.exe) but after "as" doesn't know the switch -am

olli — Thu, 13 Mar 2014 07:14:19 GMT

is already fixed

Thu, 13 Mar 2014 12:23:26 GMT

Thank you olli , indeed fixed, posted my results [1]

[1] https://svn.boost.org/trac/boost/ticket/9774#comment:4