Skip to content

conversion from Integer to Double rounds differently depending on the value

I'm not sure if this is intended behavior or a bug.

For values less than 2^53^, the the Integer can be represented exactly as a Double so there is no issue. For values from 2^53^ to 2^63^, fromInteger n rounds to the nearest Double, where ties go toward the Double with even least significant bit. This matches the behavior of read. For values between 2^63^ and 2^1024^, fromInteger n rounds down to the next representable Double.

Steps to reproduce:

These values are all the same except the last one because it falls just below the threshold for the "round to nearest, ties to even" rule. In this example, (2^63^) and (2^63^-2^10^) are the consecutive values of type Double.

> fromInteger (2^63) :: Double           -- exactly 9223372036854775808
9.223372036854776e18
> fromInteger (2^63 - 1) :: Double       -- rounded up
9.223372036854776e18
> fromInteger (2^63 - 2^9) :: Double     -- rounded up
9.223372036854776e18
> fromInteger (2^63 - 2^9 - 1) :: Double -- rounded down
9.223372036854775e18
> fromInteger (2^63 - 2^10) :: Double    -- exactly 9223372036854774784
9.223372036854775e18

Here you can see the behavior of fromInteger matches that of read . show

> (read . show) (2^63) :: Double           -- exactly 9223372036854775808
9.223372036854776e18
> (read . show) (2^63 - 1) :: Double       -- rounded up
9.223372036854776e18
> (read . show) (2^63 - 2^9) :: Double     -- rounded up
9.223372036854776e18
> (read . show) (2^63 - 2^9 - 1) :: Double -- rounded down
9.223372036854775e18
> (read . show) (2^63 - 2^10) :: Double    -- exactly 9223372036854774784
9.223372036854775e18

Here you can see fromInteger is rounding down. In this example, (2^64^) and (2^64^-2^11^) are the consecutive values of type Double.

> fromInteger (2^64) :: Double            -- exactly 18446744073709551616
1.8446744073709552e19
> fromInteger (2^64 - 1) :: Double        -- rounded down
1.844674407370955e19
> fromInteger (2^64 - 2^10) :: Double     -- rounded down
1.844674407370955e19
> fromInteger (2^64 - 2^10 - 1) :: Double -- rounded down
1.844674407370955e19
> fromInteger (2^64 - 2^11) :: Double     -- exactly 18446744073709549568
1.844674407370955e19

Here you can see the behavior of fromInteger does not match the behavior of read . show

> (read . show) (2^64) :: Double            -- exactly 18446744073709551616
1.8446744073709552e19
> (read . show) (2^64 - 1) :: Double        -- rounded up
1.8446744073709552e19
> (read . show) (2^64 - 2^10) :: Double     -- rounded up
1.8446744073709552e19
> (read . show) (2^64 - 2^10 - 1) :: Double -- rounded down
1.844674407370955e19
> (read . show) (2^64 - 2^11) :: Double     -- exactly 18446744073709549568
1.844674407370955e19
Trac metadata
Trac field Value
Version 8.6.2
Type Bug
TypeOfFailure OtherFailure
Priority normal
Resolution Unresolved
Component Compiler
Test case
Differential revisions
BlockedBy
Related
Blocking
CC
Operating system
Architecture
To upload designs, you'll need to enable LFS and have an admin enable hashed storage. More information