Version 1 (modified by 15 years ago) ( diff ) | ,
---|
Testing Boost with CMake
Boost's CMake-based build system provides regression testing via CTest, which comes as part of CMake. This regression testing system can be used by Boost developers to test their libraries locally and also by testers and users to submit regression tests to a Dart server, which collects and reports regression-testing results from different sites. This document assumes that the reader has already learned how to configure and build Boost using CMake.
- Re-run the CMake configuration for Boost, either by running the CMake configuration program from the Start menu (on Microsoft Windows) or executing the command-line CMake configuration in Unix:
ccmake $BOOST_ROOT
- Toggle the
BUILD_TESTING
option in the CMake configuration toON
, and then re-configure (either by clicking "Configure", on Microsoft Windows, or pressingc
, on Unix). You may notice that configuration takes significantly longer when we are building all of the regression tests.
- If you want, you can now disable regression testing for some or all Boost libraries by toggling the
TEST_BOOST_LIBNAME
options that appear. Be sure to re-configure CMake once you are done tweaking these options.
- Generate makefiles or project files, by clicking "OK" (on Microsoft Windows) or pressing
g
(on Unix).
- Follow the same building process described in the section Configuring and Building Boost, either be running your make program from the Boost binary directory or compiling from within your IDE (e.g., Microsoft Visual Studio). For Unix users, we suggest passing the
-i
option tomake
, and also possibly-j 2
(or more) to run the build process in parallel. Building all of the regression tests for the Boost libraries can take a long time.make -i -j 2
Note: If you change Boost source files in a way that affects your tests, you will need to re-run the build process to update the libraries and tests before moving on to the next step.
- Once regression tests have finished building, go into the command line or command prompt and enter the Boost binary directory. Then, run the command
ctest
to execute all of the regression tests. The
ctest
executable will be stored in the binary directory for CMake. On Unix platforms, this is the same place whereccmake
resides. On Windows platforms, it will be inC:\Program Files\CMake\bin
. The ctest program should produce output like the following:Start processing tests Test project /Users/dgregor/Projects/boost-darwin 1/ 22 Testing any::any_test Passed 2/ 22 Testing any::any_to_ref_test Passed 3/ 22 Testing function::lib_function_test Passed 4/ 22 Testing function::function_n_test Passed 5/ 22 Testing function::allocator_test Passed 6/ 22 Testing function::stateless_test Passed 7/ 22 Testing function::lambda_test Passed 8/ 22 Testing function::function_test_fail1 ***Failed - supposed to fail 9/ 22 Testing function::function_test_fail2 ***Failed - supposed to fail 10/ 22 Testing function::function_30 Passed 11/ 22 Testing function::function_arith_cxx98 Passed 12/ 22 Testing function::function_arith_porta Passed 13/ 22 Testing function::sum_avg_cxx98 Passed 14/ 22 Testing function::sum_avg_portable Passed 15/ 22 Testing function::mem_fun_cxx98 Passed 16/ 22 Testing function::mem_fun_portable Passed 17/ 22 Testing function::std_bind_cxx98 Passed 18/ 22 Testing function::std_bind_portable Passed 19/ 22 Testing function::function_ref_cxx98 Passed 20/ 22 Testing function::function_ref_portabl Passed 21/ 22 Testing function::contains_test Passed 22/ 22 Testing function::contains2_test Passed 100% tests passed, 0 tests failed out of 22
Here, we have only enabled testing of the Boost.Any and Boost.Function libraries, by setting
TEST_BOOST_ANY
andTEST_BOOST_FUNCTION
toON
while all of the otherTEST_BOOST_LIBNAME
options are set toOFF
.
Submitting Regression Test Results
The ctest
command can be used by individual developers to test local changes to their libraries. The same program can also be used to build all of Boost, run its regression tests, and submit the results to a central server where others can view them. Currently, regression test results based on the CMake build system are displayed on the Dart server at http://dart.resophonic.com/boost_1_34_0/Dashboard/.
To submit "experimental" results to the Dart server, configure a Boost binary tree by following the configuration instructions in the section Configuring and Building Boost and then enable regression testing via the BOOST_TESTING
option, as described above. At this point, don't build anything! We'll let CTest do that work. You may want to customize some of the advanced CMake options, such as SITE
(to give your site name), and MAKECOMMAND
(which, for makefile targets, governs the top-level make call when building Boost). Finally, go into the Boost binary directory and execute:
ctest -D Experimental
CTest will then reconfigure Boost, build all of the Boost libraries and regression tests, execute the regression tests, and submit the results to the Dart dashboard at http://dart.resophonic.com/boost_1_34_0/Dashboard/. Results submitted to the dashboard usually show up in 15-20 minutes, and can be browsed from the web site given.
Note: Although we are running regression tests on several flavors of Unix on a nightly basis, we have done very little work to ensure that regression testing runs smoothly on other platforms. We will remedy this problem in the near future.