• Daishi Nakajima's avatar
    Fix the right-shift operation for negative big integers (fixes #12136) · 06b9561a
    Daishi Nakajima authored
    In `x shiftR y`, any of the following conditions cause an abort:
    - `x` is a negative big integer
    - The size of `x` and `y` is a multiple of `GMP_NUMB_BITS`
    - The bit of the absolute value of `x` is filled with `1`
    For example:
    Assuming `GMP_NUMB_BITS = 2`,  the processing of `-15 shiftR 2` is as 
    1. -15 = -1111 (twos complement: 10001)
    2. right shift 2 (as a positive number) -> 0011
    3. Due to the shift larger than GMP_NUMB_BITS, the size of the 
    destination is decreasing (2bit) -> 11
    4. Add 1, and get carry: (1) 00
    5. abort
    I fixed it that the destination size does not decrease in such a case.
    Test Plan: I tested the specific case being reported.
    Reviewers: goldfire, austin, hvr, bgamari, rwbarton
    Reviewed By: bgamari, rwbarton
    Subscribers: mpickering, rwbarton, thomie
    Differential Revision: https://phabricator.haskell.org/D2998
    GHC Trac Issues: #12136
Type.hs 74 KB