Skip to content

Performance regression from GHC 8.10.4 to GHC 9.0.1

Summary

Two of my libraries, libarchive and phash run slower with GHC 9.0.1 as compared to 8.10.4.

Steps to reproduce

vanessa@vanessa-desktop /development/haskell/phash 🌸 cabal run phash-bench -w ghc-9.0.1
benchmarking fileHash/cat.png
time                 34.88 ms   (34.49 ms .. 35.28 ms)
                     0.999 R²   (0.998 R² .. 1.000 R²)
mean                 34.38 ms   (34.12 ms .. 34.65 ms)
std dev              598.0 μs   (483.6 μs .. 769.4 μs)

benchmarking fileHash/frog.jpeg
time                 23.71 ms   (23.48 ms .. 23.93 ms)
                     1.000 R²   (0.999 R² .. 1.000 R²)
mean                 23.37 ms   (23.21 ms .. 23.51 ms)
std dev              366.7 μs   (274.6 μs .. 489.3 μs)

benchmarking fileHash/frog.png
time                 17.64 ms   (17.50 ms .. 17.79 ms)
                     1.000 R²   (1.000 R² .. 1.000 R²)
mean                 17.62 ms   (17.56 ms .. 17.67 ms)
std dev              141.3 μs   (114.9 μs .. 172.2 μs)
vanessa@vanessa-desktop /development/haskell/phash 🌸 cabal run phash-bench -w ghc-8.10.4
benchmarking fileHash/cat.png
time                 23.30 ms   (23.16 ms .. 23.43 ms)
                     1.000 R²   (1.000 R² .. 1.000 R²)
mean                 23.36 ms   (23.27 ms .. 23.47 ms)
std dev              241.2 μs   (164.9 μs .. 363.2 μs)

benchmarking fileHash/frog.jpeg
time                 19.27 ms   (19.16 ms .. 19.37 ms)
                     1.000 R²   (1.000 R² .. 1.000 R²)
mean                 19.10 ms   (18.99 ms .. 19.17 ms)
std dev              209.3 μs   (165.7 μs .. 262.1 μs)

benchmarking fileHash/frog.png
time                 13.35 ms   (13.24 ms .. 13.48 ms)
                     1.000 R²   (0.999 R² .. 1.000 R²)
mean                 13.49 ms   (13.44 ms .. 13.54 ms)
std dev              134.4 μs   (100.1 μs .. 173.2 μs)
vanessa@vanessa-desktop /development/haskell/libarchive 🌸 cabal run libarchive-bench -w ghc-9.0.1
benchmarking roundtrip/libarchive
time                 214.0 μs   (212.2 μs .. 215.5 μs)
                     1.000 R²   (1.000 R² .. 1.000 R²)
mean                 213.1 μs   (212.4 μs .. 213.9 μs)
std dev              2.483 μs   (1.995 μs .. 3.087 μs)

benchmarking roundtrip/tar
time                 301.7 μs   (300.9 μs .. 302.9 μs)
                     1.000 R²   (1.000 R² .. 1.000 R²)
mean                 303.2 μs   (302.3 μs .. 304.4 μs)
std dev              3.475 μs   (2.790 μs .. 4.311 μs)

benchmarking unpack/libarchive (via bytestring)
time                 957.7 μs   (945.2 μs .. 969.1 μs)
                     0.999 R²   (0.998 R² .. 0.999 R²)
mean                 934.2 μs   (928.0 μs .. 943.4 μs)
std dev              26.62 μs   (19.99 μs .. 38.96 μs)
variance introduced by outliers: 18% (moderately inflated)

benchmarking unpack/libarchive (C API)
time                 902.5 μs   (887.9 μs .. 920.4 μs)
                     0.997 R²   (0.994 R² .. 1.000 R²)
mean                 911.5 μs   (903.5 μs .. 925.9 μs)
std dev              35.59 μs   (22.60 μs .. 55.99 μs)
variance introduced by outliers: 29% (moderately inflated)

benchmarking unpack/tar
time                 3.894 ms   (3.328 ms .. 4.650 ms)
                     0.836 R²   (0.756 R² .. 0.969 R²)
mean                 3.796 ms   (3.547 ms .. 4.223 ms)
std dev              1.007 ms   (685.8 μs .. 1.468 ms)
variance introduced by outliers: 93% (severely inflated)

benchmarking unpack/tarConduit
time                 3.541 ms   (3.184 ms .. 4.002 ms)
                     0.718 R²   (0.499 R² .. 0.962 R²)
mean                 3.821 ms   (3.468 ms .. 4.500 ms)
std dev              1.562 ms   (897.4 μs .. 2.804 ms)
variance introduced by outliers: 98% (severely inflated)
vanessa@vanessa-desktop /development/haskell/libarchive 🌸 cabal run libarchive-bench -w ghc-8.10.4
benchmarking roundtrip/libarchive
time                 196.5 μs   (195.9 μs .. 197.3 μs)
                     1.000 R²   (1.000 R² .. 1.000 R²)
mean                 197.2 μs   (196.7 μs .. 197.7 μs)
std dev              1.660 μs   (1.410 μs .. 2.045 μs)

benchmarking roundtrip/tar
time                 294.3 μs   (292.9 μs .. 295.7 μs)
                     1.000 R²   (1.000 R² .. 1.000 R²)
mean                 291.1 μs   (290.0 μs .. 292.3 μs)
std dev              3.735 μs   (3.192 μs .. 4.461 μs)

benchmarking unpack/libarchive (via bytestring)
time                 880.8 μs   (875.7 μs .. 886.3 μs)
                     1.000 R²   (0.999 R² .. 1.000 R²)
mean                 888.9 μs   (884.1 μs .. 894.6 μs)
std dev              17.34 μs   (13.96 μs .. 22.14 μs)

benchmarking unpack/libarchive (C API)
time                 840.6 μs   (834.9 μs .. 847.7 μs)
                     0.999 R²   (0.997 R² .. 1.000 R²)
mean                 846.6 μs   (841.3 μs .. 859.5 μs)
std dev              25.88 μs   (10.95 μs .. 52.64 μs)
variance introduced by outliers: 21% (moderately inflated)

benchmarking unpack/tar
time                 3.674 ms   (3.316 ms .. 4.108 ms)
                     0.842 R²   (0.696 R² .. 0.953 R²)
mean                 3.526 ms   (3.272 ms .. 3.958 ms)
std dev              1.072 ms   (665.0 μs .. 1.820 ms)
variance introduced by outliers: 95% (severely inflated)

benchmarking unpack/tarConduit
time                 3.672 ms   (3.453 ms .. 4.014 ms)
                     0.650 R²   (0.355 R² .. 0.976 R²)
mean                 4.023 ms   (3.632 ms .. 5.466 ms)
std dev              2.048 ms   (598.5 μs .. 4.177 ms)
variance introduced by outliers: 98% (severely inflated)

Expected behavior

Similar or better performance with GHC 9.0.1

Environment

  • GHC version used: GHC 8.10.4 and GHC 9.0.1

Optional:

  • Operating System: Linux (Ubuntu 18.04.5)
  • System Architecture: x86_64
To upload designs, you'll need to enable LFS and have an admin enable hashed storage. More information