Skip to content

Fixed format floating point conversion does not round to even

Conversion of floating point numbers to a fixed numbers of decimals should use round-to-even rather that rounding up when the number is right between two possible results.

E.g., printf "%.1f" 0.45 should produce "0.4" rather than "0.5".

The heart of the problem is in libraries/base/GHC/Float.lhs. The roundTo function should be replaced by this

roundTo :: Int -> Int -> [Int] -> (Int,[Int])
roundTo base d is =
  case f d True is of
    x@(0,_) -> x
    (1,xs)  -> (1, 1:xs)
    _       -> error "roundTo: bad Value"
 where
  b2 = base `quot` 2

  f n _ []     = (0, replicate n 0)
  f 0 e (x:xs) | x == b2 && e && all (== 0) xs = (0, [])   -- Round to even when at exactly half the base
               | otherwise = (if x >= b2 then 1 else 0, [])
  f n _ (i:xs)
     | i' == base = (1,0:ds)
     | otherwise  = (0,i':ds)
      where
       (c,ds) = f (n-1) (even i) xs
       i'     = c + i

I also (as the original author of the code) note that the large parts of that module was taken verbatim from the hbc libraries without attribution. :)

Trac metadata
Trac field Value
Version 7.4.1
Type Bug
TypeOfFailure OtherFailure
Priority normal
Resolution Unresolved
Component libraries/base
Test case
Differential revisions
BlockedBy
Related
Blocking
CC
Operating system
Architecture
To upload designs, you'll need to enable LFS and have an admin enable hashed storage. More information