Skip to content
  • Travis Whitaker's avatar
    Correct closure observation, construction, and mutation on weak memory machines. · 11bac115
    Travis Whitaker authored and Ben Gamari's avatar Ben Gamari committed
    Here the following changes are introduced:
        - A read barrier machine op is added to Cmm.
        - The order in which a closure's fields are read and written is changed.
        - Memory barriers are added to RTS code to ensure correctness on
          out-or-order machines with weak memory ordering.
    Cmm has a new CallishMachOp called MO_ReadBarrier. On weak memory machines, this
    is lowered to an instruction that ensures memory reads that occur after said
    instruction in program order are not performed before reads coming before said
    instruction in program order. On machines with strong memory ordering properties
    (e.g. X86, SPARC in TSO mode) no such instruction is necessary, so
    MO_ReadBarrier is simply erased. However, such an instruction is necessary on
    weakly ordered machines, e.g. ARM and PowerPC.
    Weam memory ordering has consequences for how closures are observed and mutated.
    For example, consider a closure that needs to be updated to an indirection. In
    order for the indirection to be safe for concurrent observers to enter, said
    observers must read the indirection's info table before they read the
    indirectee. Furthermore, the entering observer makes assumptions about the
    closure based on its info table contents, e.g. an INFO_TYPE of IND imples the
    closure has an indirectee pointer that is safe to follow.
    When a closure is updated with an indirection, both its info table and its
    indirectee must be written. With weak memory ordering, these two writes can be
    arbitrarily reordered, and perhaps even interleaved with other threads' reads
    and writes (in the absence of memory barrier instructions). Consider this
    example of a bad reordering:
    - An updater writes to a closure's info table (INFO_TYPE is now IND).
    - A concurrent observer branches upon reading the closure's INFO_TYPE as IND.
    - A concurrent observer reads the closure's indirectee and enters it. (!!!)
    - An updater writes the closure's indirectee.
    Here the update to the indirectee comes too late and the concurrent observer has
    jumped off into the abyss. Speculative execution can also cause us issues,
    - An observer is about to case on a value in closure's info table.
    - The observer speculatively reads one or more of closure's fields.
    - An updater writes to closure's info table.
    - The observer takes a branch based on the new info table value, but with the
      old closure fields!
    - The updater writes to the closure's other fields, but its too late.
    Because of these effects, reads and writes to a closure's info table must be
    ordered carefully with respect to reads and writes to the closure's other
    fields, and memory barriers must be placed to ensure that reads and writes occur
    in program order. Specifically, updates to a closure must follow the following
    - Update the closure's (non-info table) fields.
    - Write barrier.
    - Update the closure's info table.
    Observing a closure's fields must follow the following pattern:
    - Read the closure's info pointer.
    - Read barrier.
    - Read the closure's (non-info table) fields.
    This patch updates RTS code to obey this pattern. This should fix long-standing
    SMP bugs on ARM (specifically newer aarch64 microarchitectures supporting
    out-of-order execution) and PowerPC. This fixes issue #15449.
    Co-Authored-By: default avatarBen Gamari <>