Boost C++ Libraries: Ticket #994: BJam output with parallel builds jumbled.

Rob.Lievaart@… — Tue, 29 May 2007 09:41:56 GMT

Hi,

popen won't do the trick, as it only catches stdout en not stderr(AFAIK). You will need to create your own pipes for stdin,stoud and stderr, and after the fork, move these to the in/out/err filedescriptors before running execvp. (and close any other open fds). Since you would do more then call exec after the fork, vfork can no longer be used.

Then in the parent you need to have some kind of select loop waiting on the fd's for all children, and reading and collecting the output from them. And when the process has finished you can dump that to the screen. (waitexec might be the place to do that.

And on Win32 with create process, it works, eh, different. Don't know the details, for that, but createprocess is sort of a combination of fork/exec, and you can pass handles for in/out/error to the function.

Don't know about any other platforms.

Rob Lievaart

component changed

René Rivera — Tue, 29 May 2007 14:41:27 GMT

component Building Boost → bjam

kbelco@… — Tue, 29 May 2007 16:09:22 GMT

Replying to grafik: Hi Rene,

As Rob observed, that's exactly the solution I have implemented. The only concern I have is the overhead of using fork vs. vfork and the need to heap allocate memory when there's output to stderr. I've played around with only keeping the first 4k bytes of warning or error messages to avoid the heap allocation but I'm not sure which solution I prefer just yet. I've also added a signal handler (the handler is registered in jam.c) so the parent process gets notified when the child terminates.

Just wanted to let you know that this is almost done for unix. I haven't even looked at changing Win32 code.

-- Noel

Rob.Lievaart@… — Wed, 30 May 2007 09:03:41 GMT

Hi Noel,

Allways, nice to see I wasn't talking complete noncense. :-) And thanks for doing the work. My feeling is you shouldn't worry to much about the fork and memory allocation. On Linux a fork does copy on write for all memory, so in this case there should not be to much penalty. Other platforms may have similar solutions. As for the dynamic memory allocation, in general I expect very little output from the (compile) commands, only in case of errors. And compared to a fork/starting of a new process, compiling some C++ code, a few memory allocations will probably not show up at all when profiling.

I am just looking at execnt.c, and it is looking a lot more complex. (NOTE: I currently have the boost 1.33.1 sources version on this machine, so that may be a bit outdated) A lot of win95/98 support that can probably be left alone. It uses spawn instead of fork/exec. To correctly handle stdout/stderr this probably has to be replaced with CreateProcess. CreateProcess works different so this is not a trivial search/replace operation. Also reading output form the Win32 handles works differently, so it will probably have quite a big impact.

Hmm, execmac.c seems like a stub implementation, the function does (allmost) nothing.

And execvms.c simply calls system, so no asynchronous support at all.

So the last two should be easy :-) Kind Regards,

Rob Lievaart.

status changed

René Rivera — Wed, 30 May 2007 16:56:08 GMT

status new → assigned

I've applied Noel's patch for this to branch "Boost_Jam_994" of the tools/jam directory (Noel, I had to make some minor changes for portability to non-Unix platforms). I'll write up some tests to test the functionality a bit later today.

René Rivera — Thu, 31 May 2007 01:18:42 GMT

I added a tools/jam/test/parallel_actions.jam test and I'm getting two different error behaviors, when I run it on my Ubuntu Linux machine. First the output from the actions is still jumbled. And second, many times, but not always, the actions don't seem to complete. Or more likely bjam doesn't notice the actions complete. To run the test:

cd tools/jam/src
./build.sh
cd ../test
../src/bin.*/bjam -f parallel_actions.jam -j4 -sBJAM_SUBTEST=1

kbelco@… — Thu, 31 May 2007 02:45:51 GMT

Hi Rene,

I forgot to fix the output so that the command output, if any, immediately follows the command rule name.

I want the output to look like this :

darwin.compile.c++ bin.v2/libs/thread/build/darwin/release/threading-multi/read_write_mutex.o darwin.compile.c++ bin.v2/libs/thread/build/darwin/release/threading-multi/thread.o

[ any / all stdout and stderr messages for thread.o occur here ]

darwin.compile.c++ bin.v2/libs/thread/build/darwin/release/threading-multi/tss_hooks.o

[ any / all stdout and stderr messages for tss_hooks.o occur here ]

darwin.compile.c++ bin.v2/libs/thread/build/darwin/release/threading-multi/tss_dll.o darwin.compile.c++ bin.v2/libs/thread/build/darwin/release/threading-multi/tss.o

and so forth.

Is this what you were thinking?

I can replicate that problem you're seeing on my Mac so I'll work on diagnosing and fixing that first and hope that fixes the Ubuntu problem you're seeing.

kbelco@… — Fri, 01 Jun 2007 21:42:40 GMT

Replying to grafik:

I added a tools/jam/test/parallel_actions.jam test and I'm getting two different error behaviors, when I run it on my Ubuntu Linux machine. First the output from the actions is still jumbled.

This has been fixed.

And second, many times, but not always, the actions don't seem to complete. Or more likely bjam doesn't notice the actions complete.

This is fixed, no zombies and fixed some performance throughput problems.

To run the test:

I haven't run you tests but I've committed the changes to the CVS branch tag you setup. Let me know if you find any more problems.

-- Noel

René Rivera — Sat, 02 Jun 2007 03:48:59 GMT

Replying to kbelco@sandia.gov:

Replying to grafik:

I added a tools/jam/test/parallel_actions.jam test and I'm getting two different error behaviors, when I run it on my Ubuntu Linux machine. First the output from the actions is still jumbled.

This has been fixed.

It seems I was still getting the same behavior on Linux. I fixed it by adding in the dup of both stderr and stdout.

And second, many times, but not always, the actions don't seem to complete. Or more likely bjam doesn't notice the actions complete.

This is fixed, no zombies and fixed some performance throughput problems.

Yea, looks fixed.

To run the test:

I haven't run you tests but I've committed the changes to the CVS branch tag you setup. Let me know if you find any more problems.

I found some :-) And fixed them. I need to make a few more changes for the memory allocation. And then we need to figure out how to do the Windows side (yuck).

anonymous — Sat, 02 Jun 2007 05:58:58 GMT

Replying to grafik:

../src/bin.*/bjam -f parallel_actions.jam -j4 -sBJAM_SUBTEST=1

Rene,

I ran this test earlier today and it looked fine but the test jam file was modified sometime this evening and now the results look funny (some of the textual output is missing, looks like the test number).

owner, status changed

René Rivera — Sat, 02 Jun 2007 15:45:21 GMT

owner changed from René Rivera to Noel Belcourt
status assigned → new

owner changed

Noel Belcourt — Sat, 02 Jun 2007 16:28:53 GMT

owner changed from Noel Belcourt to René Rivera

status changed; resolution set

René Rivera — Sat, 16 Jun 2007 20:56:47 GMT

status new → closed
resolution → fixed