Skip to content

Performance of nofib/spectral/mandel2 is absurdly fragile

Look at this code in nofib/spectral/mandel2:

main = do
  [n] <- getArgs
  replicateM_ (read n) $ do
    -- m should always be smaller than size, but the compiler can't know that
    m <- length <$> getArgs
    let size' = max m size
    finite (build_tree (0,0) (size',size' `div` 2)) `seq` return ()

Very odd. We have just done getArgs so we know that m <- length <$> getArgs will yield 1. Hence size' = size always.

Now, if we copy that last line (an inlining-threshold decision) we end up with code like

   if m<size then
        finite (build_tree (0,0) (size,size `div` 2)) `seq` return ()
   else
        finite (build_tree (0,0) (m,m `div` 2)) `seq` return ()

Now, the first of these duplicated expressions is now a CAF and so can be floated out of the loop, and only done once -- thereby defeating whole purpose of the replicate. That happens to occur in HEAD.

But if that expression is a little bit too big, it won't be duplicated, and we won't see a CAF to lift out, and the replicate does its job. Result: execution is MUCH more expensive.

Ironically this fragility was introduced by @sgraf812 in this commit, which was intended to stabilise nofib results!

commit 8632268ad8405f0c01aaad3ad16e23c65771ba49
Author: Sebastian Graf <sebastian.graf@kit.edu>
Date:   Sun Dec 30 19:36:23 2018 +0100

    Stabilise benchmarks wrt. GC
    
    Summary:
    This is due to #15999, a follow-up on #5793 and #15357 and changes all
    benchmarks, some of them (i.e. `wheel-sieve1`, `awards`) rather drastically.
    
    The general plan is outlined in #15999: Identify GC-sensitive benchmarks by
    looking at how productivity rates change over different nursery sizes and
    iterate `main` of these benchmarks often enough for the non-monotony and
    discontinuities to go away.
    
    I was paying attention that the benchmarked logic is actually run $n times more
    often, rather than just benchmarking IO operations printing the result of CAFs.

Specifically the change was this:

-main =  if finite(build_tree (0,0) (size,size `div` 2)) then
-            print "Success"
-          else
-            print "Fail"
-
-
+main = do
+  [n] <- getArgs
+  replicateM_ (read n) $ do
+    -- m should always be smaller than size, but the compiler can't know that
+    m <- length <$> getArgs
+    let size' = max m size
+    finite (build_tree (0,0) (size',size' `div` 2)) `seq` return ()

Let's fix this fragility.

Edited by Simon Peyton Jones
To upload designs, you'll need to enable LFS and have an admin enable hashed storage. More information