Opened 10 years ago

Closed 20 months ago

#7785 closed Bugs (fixed)

karma real generator limits precision to digits10 no matter exponent

Reported by: strk@… Owned by: Hartmut Kaiser
Milestone: To Be Determined Component: spirit
Version: Boost 1.52.0 Severity: Problem
Keywords: Cc:

Description

The real_inserter::call_n method here is limiting the user-provided precision by the http://svn.boost.org/svn/boost/trunk/boost/spirit/home/karma/numeric/detail/real_utils.hpp

But that value is later used to determine how many decimal digits to print, independently on where the first non-zero digit is. This means overlimiting the actual number of significant digit for no evident reason.

The user provided ::precision() function has access to the full number and thus can judge when to provide more or less precision depending on magnitude. Why not trusting that ?

Change History (5)

comment:1 by strk@…, 10 years ago

For reference, the problem showed up here: https://github.com/mapnik/mapnik/pull/1632 where we're trying to get an output comparable with that of iostream with setprecision(16)

comment:2 by anonymous, 10 years ago

On further analysis I found out that the "fraction_part" function does not even get more than 13 significant digits when the number is lower than 10e-3, so there's nothing an overridden fraction_part method could do

comment:3 by Sandro Santilli <strk@…>, 10 years ago

The problem is in call_n, where fractional part is normalized even if the format flag is requesting fixed precision, despite the comment above (saying normalization is only needed for scientific notation).

This patch would fix my case:

--- real_utils.hpp.000  2012-12-13 13:42:01.000000000 +0100
+++ real_utils.hpp      2012-12-13 13:43:02.000000000 +0100
@@ -85,5 +85,5 @@
             // get correct precision for generated number
             unsigned precision = p.precision(n);
-            if (std::numeric_limits<U>::digits10) 
+            if (std::numeric_limits<U>::digits10 && !(Policies::fmtflags::fixed & flags))
             {
                 // limit generated precision to digits10, if defined

comment:4 by Joel de Guzman, 10 years ago

Owner: changed from Joel de Guzman to Hartmut Kaiser

comment:5 by Joel de Guzman, 20 months ago

Resolution: fixed
Status: newclosed
Note: See TracTickets for help on using tickets.