Skip to content

GitLab

Fix GHCi :print on big-endian platforms

Closed Stefan Schulze Frielinghaus requested to merge (removed):extractSubTerms-big-endian into master

On big-endian platforms executing

import GHC.Exts
data Foo = Foo Float# deriving Show
foo = Foo 42.0#
foo
:print foo

results in an arithmetic overflow exception which is caused by function index where moveBytes equals

word_size - (r + item_size_b) * 8

Here we have a mixture of units. Both, word_size and item_size_b have unit bytes whereas r has unit bits. On 64-bit platforms moveBytes equals then

8 - (0 + 4) * 8

which results in a negative and therefore invalid second parameter for a shiftL operation.

This patch fixes function index and streamlines its implementation for big-endian and little-endian platforms.

This fixes test cases print002 and print022 on s390x which also fail on PPC64be (see #16548 (closed) and #14455 (closed)).

Note1: To make things more clear, the expression

(word .&. (mask `shiftL` moveBytes)) `shiftR` moveBytes

is equivalent to

(word `shiftR` moveBytes) .&. mask

On big-endian platforms the shift must be a left shift and not a right shift. For symmetry reasons I'm not using a mask but two shifts in order to zero out bits. Thus the fixed version equals

case endian of
  BigEndian    -> (word `shiftL` moveBits) `shiftR` zeroOutBits `shiftL` zeroOutBits
  LittleEndian -> (word `shiftR` moveBits) `shiftL` zeroOutBits `shiftR` zeroOutBits

Note2: This fixes only the arithmetic overflow exception in order to make :print work again. Evaluating the expression foo still results in wrong output. In total we have:

> foo
Foo 0.0#

whereas

> :print foo
foo = Foo 42.0
Edited by Stefan Schulze Frielinghaus