Skip to content

Make perf tests less fragile -- do not rely on flukey CSE

In testsuite/tests/perf/should_run, these two tests

  • T15426
  • T18964

are accidentally very strongly affected by a CSE fluke. For example in T15426 we have this:

main = do evaluate $ L.elemIndex 999999 [(1::Int)..1000000]
          evaluate $ L.elemIndices 999999 [(1::Int)..1000000]
          evaluate $ L.findIndex (>=999999) [(1::Int)..1000000]
          evaluate $ L.findIndices (>=999999) [(1::Int)..1000000]
          evaluate $ unsafeFindIndex (>=999999) [(1::Int)..1000000]

The test is supposed to be all about fusion. But in fact if fusion doesn't happen in four of the five cases, which is why the ticket #15426 is still open. But then the lists float to top level and get combined by common subexpression elimination (CSE), which isn't the point of the test at all! It's a bit as if we had this:

xs = [(1::Int)..1000000]
main = do evaluate $ L.elemIndex 999999 xs
          evaluate $ L.elemIndices 999999 xs
          evaluate $ L.findIndex (>=999999) xs
          evaluate $ L.findIndices (>=999999) xs
          evaluate $ unsafeFindIndex (>=999999) xs

This kind of floating-then-CSE is a fluke. For example it does not happen locally. E.g.

f x = (h (g x), g x)

GHC's CSE is quite limited at the moment: it does not combine the "peer" (g x) calls in this example. So the fluke is that GHC just happens to float those constant expressions to top level and they get CSEd. As it happens I'm exploring a change that would do less floating-to-top-level, and in my MR these two perf test blew up.

So the burden of this ticket is: let's change these two tests so they are not exposed to flukey CSE changes, which are not the point of the test in the first place.

Edited by sheaf
To upload designs, you'll need to enable LFS and have an admin enable hashed storage. More information