Boost C++ Libraries: Ticket #11416: BOOST_AUTO_TEST_CASE_EXPECTED_FAILURES should result in a fail verdict if the expectation is not fullfilled https://svn.boost.org/trac10/ticket/11416 <p> Given this test: </p> <pre class="wiki">BOOST_AUTO_TEST_CASE_EXPECTED_FAILURES(fail_test, 1) BOOST_AUTO_TEST_CASE( fail_test ) { BOOST_CHECK(1 == 1); } </pre><p> I would assume that this test results in a overall failed test, but the macro only prints a message on the console: "Test case fail_test has fewer failures than expected" and does not touch the verdict of the test. This is IMHO useless, because its easy to miss (Especially if CI is used) </p> <p> I introduced that test case into our test suite to ensure that the tests are working correctly (before we had tests that did nothing and resulted in PASS, because NDEBUG was defined) </p> en-us Boost C++ Libraries /htdocs/site/boost.png https://svn.boost.org/trac10/ticket/11416 Trac 1.4.3 Raffi Enficiaud Fri, 03 Jul 2015 11:21:42 GMT status changed; resolution set https://svn.boost.org/trac10/ticket/11416#comment:1 https://svn.boost.org/trac10/ticket/11416#comment:1 <ul> <li><strong>status</strong> <span class="trac-field-old">new</span> → <span class="trac-field-new">closed</span> </li> <li><strong>resolution</strong> → <span class="trac-field-new">invalid</span> </li> </ul> <p> This is by design: an error is raised only when the number of failures inside the test case is higher than the expected one. For checking the strict number of failures, regular assertions should be used. </p> <p> This is explained in details here: <a href="http://www.boost.org/doc/libs/develop/libs/test/doc/html/boost_test/testing_tools/expected_failures.html">http://www.boost.org/doc/libs/develop/libs/test/doc/html/boost_test/testing_tools/expected_failures.html</a> </p> Ticket