Performance regression from GHC 8.10.4 to GHC 9.0.1
Summary
Two of my libraries, libarchive and phash run slower with GHC 9.0.1 as compared to 8.10.4.
Steps to reproduce
vanessa@vanessa-desktop /development/haskell/phash 🌸 cabal run phash-bench -w ghc-9.0.1
benchmarking fileHash/cat.png
time 34.88 ms (34.49 ms .. 35.28 ms)
0.999 R² (0.998 R² .. 1.000 R²)
mean 34.38 ms (34.12 ms .. 34.65 ms)
std dev 598.0 μs (483.6 μs .. 769.4 μs)
benchmarking fileHash/frog.jpeg
time 23.71 ms (23.48 ms .. 23.93 ms)
1.000 R² (0.999 R² .. 1.000 R²)
mean 23.37 ms (23.21 ms .. 23.51 ms)
std dev 366.7 μs (274.6 μs .. 489.3 μs)
benchmarking fileHash/frog.png
time 17.64 ms (17.50 ms .. 17.79 ms)
1.000 R² (1.000 R² .. 1.000 R²)
mean 17.62 ms (17.56 ms .. 17.67 ms)
std dev 141.3 μs (114.9 μs .. 172.2 μs)
vanessa@vanessa-desktop /development/haskell/phash 🌸 cabal run phash-bench -w ghc-8.10.4
benchmarking fileHash/cat.png
time 23.30 ms (23.16 ms .. 23.43 ms)
1.000 R² (1.000 R² .. 1.000 R²)
mean 23.36 ms (23.27 ms .. 23.47 ms)
std dev 241.2 μs (164.9 μs .. 363.2 μs)
benchmarking fileHash/frog.jpeg
time 19.27 ms (19.16 ms .. 19.37 ms)
1.000 R² (1.000 R² .. 1.000 R²)
mean 19.10 ms (18.99 ms .. 19.17 ms)
std dev 209.3 μs (165.7 μs .. 262.1 μs)
benchmarking fileHash/frog.png
time 13.35 ms (13.24 ms .. 13.48 ms)
1.000 R² (0.999 R² .. 1.000 R²)
mean 13.49 ms (13.44 ms .. 13.54 ms)
std dev 134.4 μs (100.1 μs .. 173.2 μs)
vanessa@vanessa-desktop /development/haskell/libarchive 🌸 cabal run libarchive-bench -w ghc-9.0.1
benchmarking roundtrip/libarchive
time 214.0 μs (212.2 μs .. 215.5 μs)
1.000 R² (1.000 R² .. 1.000 R²)
mean 213.1 μs (212.4 μs .. 213.9 μs)
std dev 2.483 μs (1.995 μs .. 3.087 μs)
benchmarking roundtrip/tar
time 301.7 μs (300.9 μs .. 302.9 μs)
1.000 R² (1.000 R² .. 1.000 R²)
mean 303.2 μs (302.3 μs .. 304.4 μs)
std dev 3.475 μs (2.790 μs .. 4.311 μs)
benchmarking unpack/libarchive (via bytestring)
time 957.7 μs (945.2 μs .. 969.1 μs)
0.999 R² (0.998 R² .. 0.999 R²)
mean 934.2 μs (928.0 μs .. 943.4 μs)
std dev 26.62 μs (19.99 μs .. 38.96 μs)
variance introduced by outliers: 18% (moderately inflated)
benchmarking unpack/libarchive (C API)
time 902.5 μs (887.9 μs .. 920.4 μs)
0.997 R² (0.994 R² .. 1.000 R²)
mean 911.5 μs (903.5 μs .. 925.9 μs)
std dev 35.59 μs (22.60 μs .. 55.99 μs)
variance introduced by outliers: 29% (moderately inflated)
benchmarking unpack/tar
time 3.894 ms (3.328 ms .. 4.650 ms)
0.836 R² (0.756 R² .. 0.969 R²)
mean 3.796 ms (3.547 ms .. 4.223 ms)
std dev 1.007 ms (685.8 μs .. 1.468 ms)
variance introduced by outliers: 93% (severely inflated)
benchmarking unpack/tarConduit
time 3.541 ms (3.184 ms .. 4.002 ms)
0.718 R² (0.499 R² .. 0.962 R²)
mean 3.821 ms (3.468 ms .. 4.500 ms)
std dev 1.562 ms (897.4 μs .. 2.804 ms)
variance introduced by outliers: 98% (severely inflated)
vanessa@vanessa-desktop /development/haskell/libarchive 🌸 cabal run libarchive-bench -w ghc-8.10.4
benchmarking roundtrip/libarchive
time 196.5 μs (195.9 μs .. 197.3 μs)
1.000 R² (1.000 R² .. 1.000 R²)
mean 197.2 μs (196.7 μs .. 197.7 μs)
std dev 1.660 μs (1.410 μs .. 2.045 μs)
benchmarking roundtrip/tar
time 294.3 μs (292.9 μs .. 295.7 μs)
1.000 R² (1.000 R² .. 1.000 R²)
mean 291.1 μs (290.0 μs .. 292.3 μs)
std dev 3.735 μs (3.192 μs .. 4.461 μs)
benchmarking unpack/libarchive (via bytestring)
time 880.8 μs (875.7 μs .. 886.3 μs)
1.000 R² (0.999 R² .. 1.000 R²)
mean 888.9 μs (884.1 μs .. 894.6 μs)
std dev 17.34 μs (13.96 μs .. 22.14 μs)
benchmarking unpack/libarchive (C API)
time 840.6 μs (834.9 μs .. 847.7 μs)
0.999 R² (0.997 R² .. 1.000 R²)
mean 846.6 μs (841.3 μs .. 859.5 μs)
std dev 25.88 μs (10.95 μs .. 52.64 μs)
variance introduced by outliers: 21% (moderately inflated)
benchmarking unpack/tar
time 3.674 ms (3.316 ms .. 4.108 ms)
0.842 R² (0.696 R² .. 0.953 R²)
mean 3.526 ms (3.272 ms .. 3.958 ms)
std dev 1.072 ms (665.0 μs .. 1.820 ms)
variance introduced by outliers: 95% (severely inflated)
benchmarking unpack/tarConduit
time 3.672 ms (3.453 ms .. 4.014 ms)
0.650 R² (0.355 R² .. 0.976 R²)
mean 4.023 ms (3.632 ms .. 5.466 ms)
std dev 2.048 ms (598.5 μs .. 4.177 ms)
variance introduced by outliers: 98% (severely inflated)
Expected behavior
Similar or better performance with GHC 9.0.1
Environment
- GHC version used: GHC 8.10.4 and GHC 9.0.1
Optional:
- Operating System: Linux (Ubuntu 18.04.5)
- System Architecture: x86_64