boost:math::float_advance( 0.0, 2 ) stack overflows if denormal numbers are set to zero

[Emmet Spier <emmet.spier@…>]

If denormal numbers are set to be zero on the processor (register MXCSR bit 6 set) then float_distance( 0.0, any number that is not 1 ) will enter invoke a function recurse which fails to exit and the program will crash with a stack overflow.

Ultimately given the current code, this is because boost::math::sign( denormal number ) == 0 as opposed to the sign of the denormal number.

However, behaviour of this function with denormal numbers really ought to be defined and documented. Possibly a test to see if denormal numbers are disabled needs to be done this case handled appropriately.

Example code, built with optimisations enabled

#include <boost/math/special_functions/next.hpp>

#include <ippcore.h>

int main() {

double thisWillWork = boost::math::float_advance( 0.0, 2 ); ippSetDenormAreZeros(1); or set MXCSR bit six directly double thisWillStackOverFlow = boost::math::float_advance( 0.0, 2 );

return 0;

}

[John Maddock]

(In [79572]) Tentative fix for issue #4445. Adds tests for next.hpp when using the SSE2 instruction set in DAZ and FTZ modes. Note that FTZ mode is particularly pernicious as non-zeros can change to zeros at random points in program flow depending upon the exact instructions generated by the compiler - thus breaking program logic/invariants. Refs #4445.

[John Maddock]

As I believe this is basically as fixed as it's possible to be (which is to say not completely!), I'm closing the issue for now.