Skip to content

GitLab

GHC
GHC
Open
Opened by Levent Erkök@LeventErkokReporter

Incorrect rounding mode used for fromIntegral :: Integer -> Double/Float

Summary

When converting from Integer to Double (or Float), GHC uses round-toward-zero rounding mode. It should use the default mode round-toward-nearest-ties-to-even instead.

It appears up till GHC 7.8.3, GHC did indeed use this default mode, but something changed after that at some point, and at least in 8.6.5, the incorrect mode is used.

Steps to reproduce

As discussed in https://stackoverflow.com/questions/58035248/a-haskell-floating-point-calculation-anomaly/58037815#58037815

This expression:

*Main> (fromIntegral $ 4141414141414141*4141414141414141) :: Double
1.7151311090705025e31

Produces an unexpected result since the resulting integer is converted using round-toward-zero. While the report is rather silent on rounding-modes, it's general accepted practice that round-nearest-ties-to-even is the default rounding mode to use. As discussed in the stack-overflow question linked, this results in bad results in diverges from other languages. For instance, the equivalent Python produces:

>>> float(long(4141414141414141)*long(4141414141414141))
1.7151311090705027e+31

which is a much better result for this expression. (Note the final digit before the exponent, 5 vs 7.)

Expected behavior

Produce the better conversion result by using round-nearest-ties-to-even rounding mode of IEEE754.

Analysis

The issue goes to this function: https://hackage.haskell.org/package/integer-gmp-1.0.2.0/docs/src/GHC.Integer.Type.html#doubleFromInteger

which reads:

doubleFromInteger :: Integer -> Double#
doubleFromInteger (S# m#) = int2Double# m#
doubleFromInteger (Jp# bn@(BN# bn#))
    = c_mpn_get_d bn# (sizeofBigNat# bn) 0#
doubleFromInteger (Jn# bn@(BN# bn#))
    = c_mpn_get_d bn# (negateInt# (sizeofBigNat# bn)) 0#

which in turn calls: https://github.com/ghc/ghc/blob/master/libraries/integer-gmp/cbits/wrappers.c#L183-L190:

/* Convert bignum to a `double`, truncating if necessary
 * (i.e. rounding towards zero).
 *
 * sign of mp_size_t argument controls sign of converted double
 */
HsDouble
integer_gmp_mpn_get_d (const mp_limb_t sp[], const mp_size_t sn,
                       const HsInt exponent)
{

This function decidedly uses round-towards-zero. Instead, a version that uses round-nearest-ties-to-even should be called.

Integer -> Float conversion suffers from the same problem, as it goes through the Double value. So, fixing the first would fix the second as well.

Environment

  • GHC version used: 8.6.5 and newer. On the stack-overflow discussion, people noted that GHC 7.8.3 returns the correct value. So, the behavior somehow changed in between these releases.

Optional:

  • Operating System: N/A
  • System Architecture: N/A
Edited by Levent Erkök