Performance of nofib/spectral/mandel2 is absurdly fragile
Look at this code in nofib/spectral/mandel2
:
main = do
[n] <- getArgs
replicateM_ (read n) $ do
-- m should always be smaller than size, but the compiler can't know that
m <- length <$> getArgs
let size' = max m size
finite (build_tree (0,0) (size',size' `div` 2)) `seq` return ()
Very odd. We have just done getArgs
so we know that m <- length <$> getArgs
will yield 1. Hence size' = size
always.
Now, if we copy that last line (an inlining-threshold decision) we end up with code like
if m<size then
finite (build_tree (0,0) (size,size `div` 2)) `seq` return ()
else
finite (build_tree (0,0) (m,m `div` 2)) `seq` return ()
Now, the first of these duplicated expressions is now a CAF and so can be floated out of the loop, and only done once -- thereby defeating whole purpose of the replicate
. That happens to occur in HEAD.
But if that expression is a little bit too big, it won't be duplicated, and we won't see a CAF to lift out, and the replicate
does its job. Result: execution is MUCH more expensive.
Ironically this fragility was introduced by @sgraf812 in this commit, which was intended to stabilise nofib results!
commit 8632268ad8405f0c01aaad3ad16e23c65771ba49
Author: Sebastian Graf <sebastian.graf@kit.edu>
Date: Sun Dec 30 19:36:23 2018 +0100
Stabilise benchmarks wrt. GC
Summary:
This is due to #15999, a follow-up on #5793 and #15357 and changes all
benchmarks, some of them (i.e. `wheel-sieve1`, `awards`) rather drastically.
The general plan is outlined in #15999: Identify GC-sensitive benchmarks by
looking at how productivity rates change over different nursery sizes and
iterate `main` of these benchmarks often enough for the non-monotony and
discontinuities to go away.
I was paying attention that the benchmarked logic is actually run $n times more
often, rather than just benchmarking IO operations printing the result of CAFs.
Specifically the change was this:
-main = if finite(build_tree (0,0) (size,size `div` 2)) then
- print "Success"
- else
- print "Fail"
-
-
+main = do
+ [n] <- getArgs
+ replicateM_ (read n) $ do
+ -- m should always be smaller than size, but the compiler can't know that
+ m <- length <$> getArgs
+ let size' = max m size
+ finite (build_tree (0,0) (size',size' `div` 2)) `seq` return ()
Let's fix this fragility.