Skip to content

showFloat for higher precision types produces strange results for some values

I've written a library which is a quda-double type a la the QD C/C++ package, and showFloat does not behave correctly for numbers with such high precision.

My type has ~212 bits of precision, and when using showFloat from Numeric, I get strange results for integer values:

show (1 :: QDouble) = "0.00000000000000000000000000000000000000000000001e47"
show (1.1 :: QDouble) = "1.1"
show (1000 :: QDouble) = "0.00000000000000000000000000000000000000000000001e50"
-- These seems to suggest it happens for any number with only a
-- few high bits set to 1 in the result of decodeFloat
show (1.125 :: QDouble) = "0.00000000000000000000000000000000000000000000001125e47"
show (1.625 :: QDouble) = "0.00000000000000000000000000000000000000000000001625e47"

The problem seems to be related to the result of floatDigits, which starts causing problem when it's larger than 56 (floatDigits x, show x):

(56,"1.0")
(57,"01.0")
(60,"001.0")
(212,"0.00000000000000000000000000000000000000000000001e47")

My fix has been to use a modified version of showFloat from Numeric by changing the floatToDigits function to include a fix for times when large numbers of zeros are produced:

  fixup2 (xs,k) = let (zs,ys) = span (==0) xs in (ys,k-length zs)
 in
 fixup2 (map fromIntegral (reverse rds), k)

This fixes the symptom but not the issue itself (though it seems like a reasonable thing to have any result returned by floatToDigits.

I have attached as minimal test case as I could come up with. Using floatToDigits from Numeric causes the strange behaviour, while floatToDigits' included in the test case does not.

Trac metadata
Trac field Value
Version 7.8.1-rc1
Type Bug
TypeOfFailure OtherFailure
Priority low
Resolution Unresolved
Component Prelude
Test case
Differential revisions
BlockedBy
Related
Blocking
CC
Operating system
Architecture
To upload designs, you'll need to enable LFS and have an admin enable hashed storage. More information