Boost C++ Libraries: Ticket #9717: Boost math library on PPC64 has thousands of errors https://svn.boost.org/trac10/ticket/9717 <p> Hi, </p> <p> I'm trying to port Boost 1.55 on PowerPC64 BE and LE. Thousands of errors appear with math library. </p> <p> Investigation one test (bessel_zeros_example_1) which loops, I see by means of gdb the following: </p> <blockquote> <p> boost::math::cyl_bessel_j_zero&lt; double, ..... boost::math::detail::cyl_bessel_j_zero_imp&lt;<span class="underline">float128, ..... </span></p> </blockquote> <p> This "<span class="underline">float128" seems erroneous. However, looking at boost/math/special_functions/bessel.hpp code, I see that this type is chosen at compilation time, by means of: </span></p> <blockquote> <p> typedef typename policies::evaluation&lt;result_type, Policy&gt;::type value_type; </p> </blockquote> <p> and it is a nightmare to understand why the compiler did choose <span class="underline">float128. Unless gdb is showing nuts ? </span></p> <p> Exact test is: bin.v2/libs/math/example/bessel_zeros_example_1.test/gcc-4.8.2/debug/bessel_zeros_example_1 </p> <p> Comparing execution path of this test code in PPC64/BE to Intel64, I see some floor() value computed differently on the 2 systems, and leading the test program to enter a wrong path and stay stucked in an infinite loop. </p> <p> There is some fundamental error, hidden by compiler which compiles templates and gives me no hint. </p> <p> I need help to determine what is wrong with Boost math on PPC64. </p> <p> Tony </p> en-us Boost C++ Libraries /htdocs/site/boost.png https://svn.boost.org/trac10/ticket/9717 Trac 1.4.3 Tony Reix <tony.reix@…> Wed, 26 Feb 2014 17:10:58 GMT <link>https://svn.boost.org/trac10/ticket/9717#comment:1 </link> <guid isPermaLink="false">https://svn.boost.org/trac10/ticket/9717#comment:1</guid> <description> <p> The issue may be due to the value of <span class="underline">LDBL_EPSILON</span> on Linux on PPC64. </p> <p> Lets use: gcc -E -dM file.cpp which prints the constants. Then: </p> <blockquote> <p> gcc -E -dM file.cpp | grep DBL_EPSILON<span class="underline"> </span></p> </blockquote> <p> gives the value of constants ending like this. </p> <p> AMD64: #define <span class="underline">LDBL_EPSILON</span> 1.08420217248550443401e-19L #define <span class="underline">DBL_EPSILON</span> double(2.22044604925031308085e-16L) </p> <p> AIX 61S PPC64: #define <span class="underline">LDBL_EPSILON</span> 2.2204460492503131e-16L #define <span class="underline">DBL_EPSILON</span> double(2.2204460492503131e-16L) </p> <p> Ubuntu/PPC64-LE: #define <span class="underline">LDBL_EPSILON</span> 4.94065645841246544176568792868221e-324L #define <span class="underline">DBL_EPSILON</span> double(2.22044604925031308084726333618164e-16L) </p> <p> Fedora/PPC64-BE: #define <span class="underline">LDBL_EPSILON</span> 4.94065645841246544176568792868221e-324L #define <span class="underline">DBL_EPSILON</span> double(2.22044604925031308084726333618164e-16L) </p> <p> IBM said: LDBL_EPSILON is the smallest nonnegative number such that 1 + LDBL_EPSILON is representable. Given that PowerPC (on Linux) uses the "double + double" representation, where the value of the long double is determined as the sum of two doubles, LDBL_EPSILON ends up very small, namely the smallest representable "double". (Because you can then represent 1 + LDBL_EPSILON by setting the first double of the pair to 1, and the second to LDBL_EPSILON.) </p> <p> The unusually small epsilon is just one aspect of the fact that the PowerPC long double format simply is not IEEE compliant. (It may well be that the fact that long double is not IEEE compliant gives trouble to Boost ... Maybe there is a way to tell Boost to avoid using long double?) </p> <p> On Intel, long double is 80-bit IEEE, which reflects the epsilon value you see there. On AIX, they're apparently using "long double" == "double". </p> <p> I think that something must be changed in Boost. However, I'm unable to find what and where. </p> </description> <category>Ticket</category> </item> <item> <dc:creator>John Maddock</dc:creator> <pubDate>Wed, 26 Feb 2014 18:02:45 GMT</pubDate> <title/> <link>https://svn.boost.org/trac10/ticket/9717#comment:2 </link> <guid isPermaLink="false">https://svn.boost.org/trac10/ticket/9717#comment:2</guid> <description> <p> The simplest solution is to define BOOST_MATH_NO_LONG_DOUBLE_MATH_FUNCTIONS which will disable all long double support. That define would go in boost/math/tools/config.hpp near the top. </p> <p> The problem with a "double double" type is that not only is it not IEEE conforming, but it's basically impossible to reason about it's behavior in any kind of sensible way. As none of us have any access to a system that uses such a type we've never tried to support it: if you wanted to try and patch Boost to support that type I have some ideas where to start, but it wouldn't be a small job I suspect :-( </p> </description> <category>Ticket</category> </item> <item> <author>Tony Reix <tony.reix@…></author> <pubDate>Thu, 27 Feb 2014 09:10:20 GMT</pubDate> <title/> <link>https://svn.boost.org/trac10/ticket/9717#comment:3 </link> <guid isPermaLink="false">https://svn.boost.org/trac10/ticket/9717#comment:3</guid> <description> <p> Replying to <a class="ticket" href="https://svn.boost.org/trac10/ticket/9717#comment:2" title="Comment 2">johnmaddock</a>: </p> <p> Thanks for your comments ! <br /> I'm now recompiling/testing Boost with such a change: </p> <p> # g++ -E -dM boost/math/tools/config.hpp | grep LONG_DOUBLE <br /> #define _GLIBCXX_LONG_DOUBLE_COMPAT 1 <br /> #define <span class="underline">SIZEOF_LONG_DOUBLE</span> 16 <br /> #define BOOST_MATH_NO_NATIVE_LONG_DOUBLE_FP_CLASSIFY <br /> <strong>#define BOOST_MATH_NO_LONG_DOUBLE_MATH_FUNCTIONS</strong> <br /> #define <span class="underline">LONG_DOUBLE_128</span> 1 <br /> #define <span class="underline">LONG_DOUBLE_MATH_OPTIONAL 1 <br /> </span></p> <p> I should have the answer in some hours. </p> <p> About your suggestion to provide me with some ideas on how to support "double double", I'm OK to work with you. I may spend days and maybe some weeks on this. I have access to both PPC64 BIG/LITTLE Endian machines, installed either with Ubuntu or Fedora (for now). <br /> I'm new to Boost. <br /> Just let me know your suggestions, plus some explanations and pages to read, and I'll experiment them. </p> <p> :) </p> </description> <category>Ticket</category> </item> <item> <dc:creator>John Maddock</dc:creator> <pubDate>Thu, 27 Feb 2014 12:15:39 GMT</pubDate> <title/> <link>https://svn.boost.org/trac10/ticket/9717#comment:4 </link> <guid isPermaLink="false">https://svn.boost.org/trac10/ticket/9717#comment:4</guid> <description> <p> Actually I see there's at least a partial patch for "double double" support in there already, it's in boost/math/tools/precision.hpp: </p> <pre class="wiki">#if (defined(macintosh) || defined(__APPLE__) || defined(__APPLE_CC__)) &amp;&amp; ((LDBL_MANT_DIG == 106) || (__LDBL_MANT_DIG__ == 106)) template &lt;&gt; inline long double epsilon&lt;long double&gt;(const mpl::true_&amp; BOOST_MATH_APPEND_EXPLICIT_TEMPLATE_TYPE(long double)) { // numeric_limits on Darwin tells lies here. // This static assert fails for some unknown reason, so // disabled for now... // BOOST_STATIC_ASSERT(std::numeric_limits&lt;long double&gt;::digits == 106); return 2.4651903288156618919116517665087e-32L; } #endif </pre><p> So that preprocessor branch need to be enabled for your platform as well. Then it's a case of seeing what still breaks. </p> <p> Are you familiar with Git? If so the best thing would be for you to clone Boost.Math develop branch (<a class="ext-link" href="https://github.com/boostorg/math/tree/develop"><span class="icon">​</span>https://github.com/boostorg/math/tree/develop</a>) and submit pull requests. </p> <p> Cheers, John. </p> </description> <category>Ticket</category> </item> <item> <author>Tony Reix <tony.reix@…></author> <pubDate>Thu, 27 Feb 2014 14:05:37 GMT</pubDate> <title/> <link>https://svn.boost.org/trac10/ticket/9717#comment:5 </link> <guid isPermaLink="false">https://svn.boost.org/trac10/ticket/9717#comment:5</guid> <description> <p> Replying to <a class="ticket" href="https://svn.boost.org/trac10/ticket/9717#comment:4" title="Comment 4">johnmaddock</a>: </p> <p> I've the result for the previous change (BOOST_MATH_NO_LONG_DOUBLE_MATH_FUNCTIONS): I now have MUCH less errors. However, I simply did a quick rebuild/retest from existing state and not a complete rebuilt/retest after a full clean . Anyway, I now see that this long double issue is the root cause for MANY of the errors I've seen. </p> <p> About your second suggestion, I've removed the previous change (NO "long double") and added a defined(<span class="underline">PPC</span>) at beginning of the #if . However, I have yet no idea about how long is the mantissa... I've asked for some paper summarizing the choices made for Linux on PPC64 (because they are different from the ones made for AIX). </p> <p> About github, I'm no more familiar, but I have a colleague who knows it very well and should be able to refresh/update my memory. </p> <p> Last: are there some Boost documentation explaining the main files used by Boost for its basic configuration, like boost/math/tools/config.hpp ? </p> <p> <a class="missing wiki">Thanks/Merci</a> </p> <p> Tony </p> </description> <category>Ticket</category> </item> <item> <author>Tony Reix <tony.reix@…></author> <pubDate>Thu, 27 Feb 2014 14:12:42 GMT</pubDate> <title/> <link>https://svn.boost.org/trac10/ticket/9717#comment:6 </link> <guid isPermaLink="false">https://svn.boost.org/trac10/ticket/9717#comment:6</guid> <description> <p> #define <span class="underline">LDBL_MAX</span> 1.79769313486231580793728971405301e+308L <br /> #define <span class="underline">LDBL_MAX_EXP</span> 1024 <br /> #define <span class="underline">LDBL_HAS_INFINITY</span> 1 <br /> #define <span class="underline">LDBL_MIN</span> 2.00416836000897277799610805135016e-292L <br /> #define <span class="underline">LDBL_HAS_QUIET_NAN</span> 1 <br /> #define <span class="underline">LDBL_HAS_DENORM</span> 1 <br /> #define <span class="underline">LDBL_EPSILON</span> 4.94065645841246544176568792868221e-324L <br /> #define <span class="underline">LDBL_MANT_DIG</span> 106 <br /> #define <span class="underline">LDBL_MIN_EXP</span> (-968) <br /> #define <span class="underline">LDBL_DENORM_MIN</span> 4.94065645841246544176568792868221e-324L <br /> #define <span class="underline">LDBL_MIN_10_EXP</span> (-291) <br /> #define <span class="underline">LDBL_DIG</span> 31 <br /> </p> </description> <category>Ticket</category> </item> <item> <author>Tony Reix <tony.reix@…></author> <pubDate>Thu, 27 Feb 2014 16:34:19 GMT</pubDate> <title/> <link>https://svn.boost.org/trac10/ticket/9717#comment:7 </link> <guid isPermaLink="false">https://svn.boost.org/trac10/ticket/9717#comment:7</guid> <description> <p> Replying to <a class="ticket" href="https://svn.boost.org/trac10/ticket/9717#comment:4" title="Comment 4">johnmaddock</a>: </p> <blockquote class="citation"> <p> Are you familiar with Git? If so the best thing would be for you to clone Boost.Math develop branch (<a class="ext-link" href="https://github.com/boostorg/math/tree/develop"><span class="icon">​</span>https://github.com/boostorg/math/tree/develop</a>) and submit pull requests. </p> </blockquote> <p> I have now a clone of Boost.Math develop branch on my PC.<br /> What can I do with it ? I even don't know how to compile it. </p> <p> Thx Tony </p> </description> <category>Ticket</category> </item> <item> <author>Tony Reix <tony.reix@…></author> <pubDate>Thu, 27 Feb 2014 17:24:59 GMT</pubDate> <title/> <link>https://svn.boost.org/trac10/ticket/9717#comment:8 </link> <guid isPermaLink="false">https://svn.boost.org/trac10/ticket/9717#comment:8</guid> <description> <p> Replying to <a class="ticket" href="https://svn.boost.org/trac10/ticket/9717#comment:4" title="Comment 4">johnmaddock</a>: </p> <p> About your seconde proposal (I've removed the previous change (NO "long double") and I've added a defined(PPC) at beginning of the #if in boost/math/tools/precision.hpp), I still have errors, but much less than at first. And, more important, I no more have the tests looping infinitely and no core in math library. </p> <p> 4300 tests (whole Boost) <br /> 83 failures <br /> </p> <blockquote> <p> (inc. 48 failure in boost/math library) <br /> </p> </blockquote> <p> Only 7046 failures in the sub-tests (instead of 50,000) </p> <p> That seems to be a good start ! </p> <p> Here are the failing math tests: </p> <p> 161 test_tr1_c_long_double.test/gcc-4.8.2/debug/link-static/test_tr1_c_long_double.output <br /> 201 complex_test.test/gcc-4.8.2/debug/link-static/complex_test.output <br /> 201 octonion_test.test/gcc-4.8.2/debug/link-static/octonion_test.output <br /> 201 quaternion_test.test/gcc-4.8.2/debug/link-static/quaternion_test.output <br /> 201 test_airy.test/gcc-4.8.2/debug/link-static/test_airy.output <br /> 201 test_bessel_airy_zeros.test/gcc-4.8.2/debug/link-static/test_bessel_airy_zeros.output <br /> 201 test_bessel_j.test/gcc-4.8.2/debug/link-static/test_bessel_j.output <br /> 201 test_bessel_k.test/gcc-4.8.2/debug/link-static/test_bessel_k.output <br /> 201 test_bessel_y.test/gcc-4.8.2/debug/link-static/test_bessel_y.output <br /> 201 test_beta.test/gcc-4.8.2/debug/link-static/test_beta.output <br /> 201 test_binomial_coeff.test/gcc-4.8.2/debug/link-static/test_binomial_coeff.output <br /> 201 test_binomial_real_concept1.test/gcc-4.8.2/debug/link-static/test_binomial_real_concept1.output <br /> 201 test_binomial_real_concept2.test/gcc-4.8.2/debug/link-static/test_binomial_real_concept2.output <br /> 201 test_binomial_real_concept4.test/gcc-4.8.2/debug/link-static/test_binomial_real_concept4.output <br /> 201 test_binomial_real_concept6.test/gcc-4.8.2/debug/link-static/test_binomial_real_concept6.output <br /> 201 test_constants.test/gcc-4.8.2/debug/link-static/test_constants.output <br /> 201 test_expint.test/gcc-4.8.2/debug/link-static/test_expint.output <br /> 201 test_factorials.test/gcc-4.8.2/debug/link-static/test_factorials.output <br /> 201 test_hankel.test/gcc-4.8.2/debug/link-static/test_hankel.output <br /> 201 test_hypergeometric_dist0.test/gcc-4.8.2/debug/link-static/test_hypergeometric_dist0.output <br /> 201 test_hypergeometric_dist1.test/gcc-4.8.2/debug/link-static/test_hypergeometric_dist1.output <br /> 201 test_hypergeometric_dist2.test/gcc-4.8.2/debug/link-static/test_hypergeometric_dist2.output <br /> 201 test_hypergeometric_dist3.test/gcc-4.8.2/debug/link-static/test_hypergeometric_dist3.output <br /> 201 test_hypergeometric_dist4.test/gcc-4.8.2/debug/link-static/test_hypergeometric_dist4.output <br /> 201 test_hypergeometric_dist5.test/gcc-4.8.2/debug/link-static/test_hypergeometric_dist5.output <br /> 201 test_ibeta_double.test/gcc-4.8.2/debug/link-static/test_ibeta_double.output <br /> 201 test_ibeta_inv_ab_double.test/gcc-4.8.2/debug/link-static/test_ibeta_inv_ab_double.output <br /> 201 test_ibeta_inv_double.test/gcc-4.8.2/debug/link-static/test_ibeta_inv_double.output <br /> 201 test_ibeta_inv_long_double.test/gcc-4.8.2/debug/link-static/test_ibeta_inv_long_double.output <br /> 201 test_ibeta_real_concept3.test/gcc-4.8.2/debug/link-static/test_ibeta_real_concept3.output <br /> 201 test_igamma_inva_real_concept.test/gcc-4.8.2/debug/link-static/test_igamma_inva_real_concept.output <br /> 201 test_igamma_inv_real_concept.test/gcc-4.8.2/debug/link-static/test_igamma_inv_real_concept.output <br /> 201 test_jacobi.test/gcc-4.8.2/debug/link-static/test_jacobi.output <br /> 201 test_legendre.test/gcc-4.8.2/debug/link-static/test_legendre.output <br /> 201 test_nc_beta_double.test/gcc-4.8.2/debug/link-static/test_nc_beta_double.output <br /> 201 test_nc_beta_long_double.test/gcc-4.8.2/debug/link-static/test_nc_beta_long_double.output <br /> 201 test_nc_chi_squared_long_double.test/gcc-4.8.2/debug/link-static/test_nc_chi_squared_long_double.output <br /> 201 test_nc_chi_squared_real_concept.test/gcc-4.8.2/debug/link-static/test_nc_chi_squared_real_concept.output <br /> 201 test_nc_t_long_double.test/gcc-4.8.2/debug/link-static/test_nc_t_long_double.output <br /> 201 test_next.test/gcc-4.8.2/debug/link-static/test_next.output <br /> 201 test_owens_t.test/gcc-4.8.2/debug/link-static/test_owens_t.output <br /> 201 test_rationals.test/gcc-4.8.2/debug/link-static/test_rationals.output <br /> 201 test_round.test/gcc-4.8.2/debug/link-static/test_round.output <br /> 201 test_skew_normal.test/gcc-4.8.2/debug/link-static/test_skew_normal.output <br /> 201 test_spherical_harmonic.test/gcc-4.8.2/debug/link-static/test_spherical_harmonic.output <br /> 201 test_students_t.test/gcc-4.8.2/debug/link-static/test_students_t.output <br /> 201 test_tr1_long_double.test/gcc-4.8.2/debug/link-static/test_tr1_long_double.output <br /> 201 test_zeta.test/gcc-4.8.2/debug/link-static/test_zeta.output <br /> </p> <p> Let me know if you have a recommendation about which one I should look at at first. </p> <p> Thx ! Tony </p> </description> <category>Ticket</category> </item> <item> <author>Tony Reix <tony.reix@…></author> <pubDate>Wed, 05 Mar 2014 09:23:45 GMT</pubDate> <title/> <link>https://svn.boost.org/trac10/ticket/9717#comment:9 </link> <guid isPermaLink="false">https://svn.boost.org/trac10/ticket/9717#comment:9</guid> <description> <p> Discussion is continuing by exchanging emails. </p> <p> IBM implements "long double" as 2 doubles (128 bits). It seems that Boost does not (or not entirely) manage this case. </p> <p> John and I are working together to determine how to have "long double" correctly managed on Linux On Power. </p> </description> <category>Ticket</category> </item> <item> <author>Tony Reix <tony.reix@…></author> <pubDate>Wed, 12 Mar 2014 16:35:26 GMT</pubDate> <title/> <link>https://svn.boost.org/trac10/ticket/9717#comment:10 </link> <guid isPermaLink="false">https://svn.boost.org/trac10/ticket/9717#comment:10</guid> <description> <p> Hi John Are you back ? so that we continue experimenting and fixing this issue ? Thx Tony </p> </description> <category>Ticket</category> </item> </channel> </rss>