Opened 10 years ago
Closed 20 months ago
#7785 closed Bugs (fixed)
karma real generator limits precision to digits10 no matter exponent
Reported by: | Owned by: | Hartmut Kaiser | |
---|---|---|---|
Milestone: | To Be Determined | Component: | spirit |
Version: | Boost 1.52.0 | Severity: | Problem |
Keywords: | Cc: |
Description
The real_inserter::call_n method here is limiting the user-provided precision by the http://svn.boost.org/svn/boost/trunk/boost/spirit/home/karma/numeric/detail/real_utils.hpp
But that value is later used to determine how many decimal digits to print, independently on where the first non-zero digit is. This means overlimiting the actual number of significant digit for no evident reason.
The user provided ::precision() function has access to the full number and thus can judge when to provide more or less precision depending on magnitude. Why not trusting that ?
Change History (5)
comment:1 by , 10 years ago
comment:2 by , 10 years ago
On further analysis I found out that the "fraction_part" function does not even get more than 13 significant digits when the number is lower than 10e-3, so there's nothing an overridden fraction_part method could do
comment:3 by , 10 years ago
The problem is in call_n, where fractional part is normalized even if the format flag is requesting fixed precision, despite the comment above (saying normalization is only needed for scientific notation).
This patch would fix my case:
--- real_utils.hpp.000 2012-12-13 13:42:01.000000000 +0100 +++ real_utils.hpp 2012-12-13 13:43:02.000000000 +0100 @@ -85,5 +85,5 @@ // get correct precision for generated number unsigned precision = p.precision(n); - if (std::numeric_limits<U>::digits10) + if (std::numeric_limits<U>::digits10 && !(Policies::fmtflags::fixed & flags)) { // limit generated precision to digits10, if defined
comment:4 by , 10 years ago
Owner: | changed from | to
---|
comment:5 by , 20 months ago
Resolution: | → fixed |
---|---|
Status: | new → closed |
For reference, the problem showed up here: https://github.com/mapnik/mapnik/pull/1632 where we're trying to get an output comparable with that of iostream with setprecision(16)