DmdAnal: Do not only preserve precise exceptions, but also externally visible side-effects

Consider

import Control.Concurrent.MVar

data Box = Box ()

f :: Box -> Int -> (Box -> Int -> Int) -> MVar Int -> IO a
f x@(Box _) n g var = do
  let m = g x n
  putMVar var m -- substitute with any side-effect
  f x (n+1) g var

main :: IO a
main = newEmptyMVar >>= f (Box (error "boom")) 100 (\_ n -> n)

This is a play on #20111. If you compile this piece of code with -O1, main will be optimised to a crash, thus swallowing the side-effecting loop. With -O0, you can observe the putMVar from concurrent threads (say).

Clearly, this optimisation is unsound given any semantics that incorporates multi-threading and assuming that the putMVar write will eventually be flushed to shared memory.

It is also easy to fix, and I did so in ce64b397. But that led to #17653 (closed), where we regressed real-world code that assumed that a side-effecting memory write is strict in its continuation. Which of course it can't be; if I replace putMVar with e.g., a call to writeArray (the particular situation in #17653 (closed) had a call to the writeWord8OffAddr#), I still expect to get an infinite loop that writes to the array instead of a instantaneous crash without any write.

Simon writes:

I agree that there is a tension between compiling efficient code and being semantically predictable.

We are in a terrible place here where GHC optimises unsoundly, but it appears that real-world code relies on that unsound optimisation.

Simon also says

It's clearly a bit of a tension, but if we document it carefully (including why -- poster-child examples of an inner loop where we want strictness analysis to work), I think it'll be fine.

I think we that we should only do this change when we can monitor its impact on downstream libraries like bytestring, vector or base64-bytestring.