https://svn.boost.org/trac10/ticket/1560 <p> I have been trying to use boost::test to do some performance testing. </p> <p> With the following macros: </p> <pre class="wiki">// The basic idea of "test" is to call the function to be timed many // times (say up to 1000), throw away the slowest 10% of those times, // and average the rest. Why? &lt;shrug&gt; shoot-from-the-hip "theory" and // experience that its return is fairly consistent to 2, maybe 3 // digits. The test function has a "time out" feature where it quits if // the accumulated measured time grows beyond 1 second (which may not be // appropriate for all tests). But it is easy to get bored when you // unexpectedly find yourself waiting 30 minutes for timing result, so // that's why it's there. Otoh, I put in a minimum repetition of at // least 10, no matter how long the measured time is, so you get at // least some accuracy (tweak knobs as you like). Note that the // accumulation/averaging happens in double, even though the data and // answer are float (just paranoia really). Weakness: If you're timing // something that is quicker than the minimum resolution of your timer, // this doesn't work. But otherwise, this is better than the // traditional loop inside the timer as it throws away those results // that happen to get interrupted by your email checker running. :-) #include &lt;numeric&gt; #include &lt;boost\timer.hpp&gt; template &lt;class F&gt; double time_tests(F f) // f is a function that returns its execution time { std::vector&lt;double&gt; t; // Store time of 10 executions of f unsigned int i; for (i = 0; i &lt; 10; ++i) t.push_back(f()); double total_time = std::accumulate(t.begin(), t.end(), 0.0); // Keep running until at least 1s of results are available, or 1000 executions while (i &lt; 1000 &amp;&amp; total_time &lt; 1.0) { t.push_back(f()); total_time += t.back(); ++i; } std::sort(t.begin(), t.end()); t.resize(t.size() * 9 / 10); return std::accumulate(t.begin(), t.end(), 0.0) / t.size(); } #define TIMED_TEST_CASE( test_name ) \ double \ time_test_##test_name() \ { \ boost::timer t; \ { \ test_name##_impl(); \ } \ return t.elapsed(); \ } #define TIMED_AUTO_TEST_CASE( test_name ) \ void \ test_name##_impl(); \ \ TIMED_TEST_CASE( test_name ) \ \ BOOST_AUTO_TEST_CASE( test_name ) \ { \ double execution_time = time_test_##test_name(); \ boost::unit_test::unit_test_log.set_threshold_level( boost::unit_test::log_messages ); \ BOOST_TEST_MESSAGE(BOOST_TEST_STRINGIZE( test_name ).trim( "\"" ) &lt;&lt; " execution time: " &lt;&lt; execution_time &lt;&lt; "s"); \ BOOST_CHECK( true ); \ } \ \ inline void test_name##_impl() </pre><p> I can define tests such as </p> <pre class="wiki"> // Boost.Test //#define BOOST_AUTO_TEST_MAIN // #include &lt;boost/test/unit_test.hpp&gt; #define BOOST_TEST_MODULE allTests #include &lt;boost/test/unit_test.hpp&gt; #include "time_test.h" #include &lt;vector&gt; #define BaseT float BOOST_AUTO_TEST_SUITE(vectors); TIMED_AUTO_TEST_CASE( vector_test ) { unsigned int const v1_dim = 6; unsigned int const v2_dim = 4; unsigned int const v3_dim = 65535; std::vector&lt;BaseT&gt; v1(v1_dim, 1.0); std::vector&lt; std::vector&lt;BaseT&gt; &gt; v2(v2_dim, v1); std::vector&lt; std::vector&lt; std::vector&lt;BaseT&gt; &gt; &gt; v3(v3_dim, v2); } TIMED_AUTO_TEST_CASE( test2 ) { BOOST_CHECK( true ); } TIMED_AUTO_TEST_CASE( test3 ) { for (int i=0; i&lt;10000; i++) { BOOST_CHECK( true ); } } BOOST_AUTO_TEST_SUITE_END(); </pre><p> This works, but is not particularly elegant. Is there a better solution (eg, involving defining a class that inherits from one of the existing ones)? </p> en-us /htdocs/site/boost.png https://svn.boost.org/trac10/ticket/1560 Trac 1.4.3 Gennadiy Rozental Mon, 21 Jan 2008 17:14:18 GMT https://svn.boost.org/trac10/ticket/1560#comment:1 https://svn.boost.org/trac10/ticket/1560#comment:1 <ul> <li><strong>status</strong> <span class="trac-field-old">new</span> → <span class="trac-field-new">closed</span> </li> <li><strong>type</strong> <span class="trac-field-old">Bugs</span> → <span class="trac-field-new">Feature Requests</span> </li> <li><strong>resolution</strong> → <span class="trac-field-new">wontfix</span> </li> </ul> <p> These macros have number of issues: </p> <ol><li>Problem domain is unclear. What exactly are you trying to test? </li><li>Number of parameters are using. none of them is configurable. It's unacceptable in generic solution </li><li>boost:;timer is not a best device for use to measure performace. </li></ol><p> I do plan to introduce some performace testing toosl. But these macro doesn't look right right way to go </p> Ticket jrp at dial dot pipex dot com Mon, 21 Jan 2008 19:54:47 GMT https://svn.boost.org/trac10/ticket/1560#comment:2 https://svn.boost.org/trac10/ticket/1560#comment:2 <ul> <li><strong>status</strong> <span class="trac-field-old">closed</span> → <span class="trac-field-new">reopened</span> </li> <li><strong>resolution</strong> <span class="trac-field-deleted">wontfix</span> </li> </ul> <blockquote class="citation"> <blockquote> <p> These macros have number of issues: </p> </blockquote> <ol><li>Problem domain is unclear. What exactly are you trying to test? </li></ol></blockquote> <p> I am just trying to do some array manipulation using a range of different packages. </p> <blockquote class="citation"> <ol start="2"><li>Number of parameters are using. none of them is configurable. It's unacceptable in generic solution </li></ol></blockquote> <p> That is also a problem for me, as I would like to try different array lengths. </p> <blockquote class="citation"> <ol start="3"><li>boost:;timer is not a best device for use to measure performace. </li></ol></blockquote> <p> Yes. </p> <blockquote class="citation"> <blockquote> <p> I do plan to introduce some performace testing toosl. But these macro doesn't look right right way to go </p> </blockquote> </blockquote> <p> I agree. What I need is something that: </p> <ul><li>time the running of a test case (that is already there, if the right logging is enabled, although the output is messy) </li><li>can distinguish between fixture setup time and time to run the test </li><li>can be set to count the number of times that a test must be run before a specified period elapses </li><li>give an indication of whether the test takes a linear amount of time with respect to the test parameter (eg, the length of the array) </li></ul> Ticket Gennadiy Rozental Wed, 23 Jan 2008 07:06:57 GMT