Skip to content
  • Daishi Nakajima's avatar
    Fix the right-shift operation for negative big integers (fixes #12136) · 06b9561a
    Daishi Nakajima authored and Ben Gamari's avatar Ben Gamari committed
    In `x shiftR y`, any of the following conditions cause an abort:
    - `x` is a negative big integer
    - The size of `x` and `y` is a multiple of `GMP_NUMB_BITS`
    - The bit of the absolute value of `x` is filled with `1`
    
    For example:
    Assuming `GMP_NUMB_BITS = 2`,  the processing of `-15 shiftR 2` is as 
    follows:
    
    1. -15 = -1111 (twos complement: 10001)
    2. right shift 2 (as a positive number) -> 0011
    3. Due to the shift larger than GMP_NUMB_BITS, the size of the 
    destination is decreasing (2bit) -> 11
    4. Add 1, and get carry: (1) 00
    5. abort
    
    I fixed it that the destination size does not decrease in such a case.
    
    Test Plan: I tested the specific case being reported.
    
    Reviewers: goldfire, austin, hvr, bgamari, rwbarton
    
    Reviewed By: bgamari, rwbarton
    
    Subscribers: mpickering, rwbarton, thomie
    
    Differential Revision: https://phabricator.haskell.org/D2998
    
    GHC Trac Issues: #12136
    06b9561a