A benchmark compiled with ghc-9.5.20220606 is 54% slower than with 9.2.3 (and uses more memory); another benchmark is 18% slower with HEAD than 9.0.2 (but twice faster than with 9.2.3)
Edit: this is going to be closed in favour of several split and updated tickets. See comments at the end.
This is about runtime performance and memory use of compiled programs. Details and recorded test data are at https://github.com/Mikolaj/horde-ad/issues/14#issuecomment-1150812589. Please disregard mentions of !7847 (closed), because the regressions happen also without !7847 (closed). I'm copying here reproduction instructions for both benchmarks in question:
git clone git@github.com:Mikolaj/horde-ad.git
git checkout ghc-report-specialize
- build HEAD with Hadrian
- add
cabal.project.local
file with (see http://ghc.gitlab.haskell.org/head.hackage)
repository head.hackage.ghc.haskell.org
url: https://ghc.gitlab.haskell.org/head.hackage/
secure: True
key-threshold: 3
root-keys:
7541f32a4ccca4f97aea3b22f5e593ba2c0267546016b992dfadcd2fe944e55d
26021a13b401500c8eb2761ca95c61f2d625bfef951b939a8124ed12ecf07329
f76d08be13e9a61a377a85e2fb63f4c5435d40f8feb3e12eb05905edb8cdea89
cabal update
cabal build -w /path_to_ghc/_build/stage1/bin/ghc --allow-newer --enable-benchmarks
- For the first benchmark (called BENCH1), run the following, substituting a correct path of the binary:
/home/mikolaj/r/horde-ad/dist-newstyle/build/x86_64-linux/ghc-9.5.20220608/horde-ad-0.1.0.0/b/mnist/build/mnist/mnist -n 10 -m prefix "2-hidden-layer MNIST nn with samples: 500/test 500" +RTS -s
- For the second benchmark (called BENCH2), run the following, substituting a correct path of the binary:
/home/mikolaj/r/horde-ad/dist-newstyle/build/x86_64-linux/ghc-9.5.20220608/horde-ad-0.1.0.0/b/mnist/build/mnist/mnist -n 1 -m prefix "2-hidden-layer MNIST nn with samples: 500/500" +RTS -s