Erroneous return value propagation in Boost.Build

[Jason Kovacs]

Hello,

I would like to report a problem I have noticed using Boost.Build combined with the Boost Unit Testing Framework. The problem manifests when using the run rule to execute unit tests, in a fashion similar to what is shown on this ​tutorial.

The problem is that, on Windows platforms, when one or more tests fail, the bjam return code is 0, while it is 1 on Linux. The propagation of this erroneous return value has been traced to the following code snippet, from tools/build/v2/tools/testing.jam:

actions capture-output bind INPUT_FILES output-file
{
    $(PATH_SETUP)
    $(LAUNCHER) "$(>)" $(ARGS) "$(INPUT_FILES)" > "$(output-file)" 2>&1
    $(.SET_STATUS)
    $(.RUN_OUTPUT_NL) >> "$(output-file)"
    echo EXIT STATUS: $(.STATUS) >> "$(output-file)"
    if $(.STATUS_0)
        $(.CP) "$(output-file)" "$(<)"
    $(.ENDIF)
    $(.SHELL_SET)verbose=$(.VERBOSE_TEST)
    if $(.STATUS_NOT_0)
        $(.SHELL_SET)verbose=1
    $(.ENDIF)
    if $(.VERBOSE)
        echo ====== BEGIN OUTPUT ======
        $(.CATENATE) "$(output-file)"
        echo ====== END OUTPUT ======
    $(.ENDIF)
    exit $(.STATUS)
}

The problem comes from the exit command, which, called from a Windows batch file, closes the whole shell instead of the batch file, causing %ERRORLEVEL% to be reset to 0. The proper way to call exit from a batch file is the following:

exit /B $(.STATUS)

For more information, type exit /? in a Windows command shell. For now, the error level propagated is always zero, which is not consistent with the Linux implementation. A diff attempting at correcting this particular problem was submitted with this post.

---

Also, this is a good opportunity to discuss what return value bjam should return when the build is over. It makes sense to return 0 for a successful build, and 1 if some targets could not be built.

But, on the other hand, it may not be desirable to have a non-zero return value because an executable launched by the run rule has exited with a non-null return value. Some continuous integration tools such as ​Pulse use bjam's return value to determine whether the build was successful or not. In test-driven development conditions, unit tests are meant to fail until issues are resolved. The fact that the unit tests have a non-zero return value does not indicate a build failure - the build may actually have succeeded.

Therefore, would it be possible to have bjam return 0 even though one of the programs called through the run rule has a non-zero exit code? Or could this feature at least be switched on with a flag?

Thanks in advance.

[Jason Kovacs]

Fix suggestion for this problem

[Steven Watanabe]

I can't reproduce this problem. Exiting cmd.exe is perfectly fine because Boost.Build creates a separate shell for each updating action.