Investigate compiler performance of Distribution.SPDX.LicenseId
I was recently using Cabal (commit e8277ee2916ca161fb80cb7fba57942290e62244) compilation as a compiler benchmark and noticed that the Distribution.SPDX.LicenseId causes GHC an enormous amount of grief. We produce incredible amounts of code and the 2kLoC module takes nearly 30 seconds to compile with -O (11 seconds with -O0).
This is somewhat understandable as the module is a bit of a perfect storm when it comes to compiler performance containing:
- a massive 500 constructor enumeration
- lots of GHC derived instances including
Enum,Generic, andData - several instances derived via
Generic(includingBinary) - several functions with complete pattern-matches on the enumeration
It is surely a sign that the -ddump-simpl output is over 340MBytes. We can no doubt do much better here.
`ghc -O -v` output from GHC 8.6.4
[18 of 18] Compiling Distribution.SPDX.LicenseId ( Distribution/SPDX/LicenseId.hs, Distribution/SPDX/LicenseId.o )
*** Parser [Distribution.SPDX.LicenseId]:
!!! Parser [Distribution.SPDX.LicenseId]: finished in 28.16 milliseconds, allocated 47.489 megabytes
*** Renamer/typechecker [Distribution.SPDX.LicenseId]:
!!! Renamer/typechecker [Distribution.SPDX.LicenseId]: finished in 2982.74 milliseconds, allocated 2837.029 megabytes
*** Desugar [Distribution.SPDX.LicenseId]:
Result size of Desugar (before optimization)
= {terms: 37,410,
types: 3,297,715,
coercions: 7,640,
joins: 0/4,431}
Result size of Desugar (after optimization)
= {terms: 32,454,
types: 3,261,682,
coercions: 15,599,
joins: 0/2,170}
!!! Desugar [Distribution.SPDX.LicenseId]: finished in 771.55 milliseconds, allocated 409.025 megabytes
*** Simplifier [Distribution.SPDX.LicenseId]:
Result size of Simplifier iteration=1
= {terms: 46,169,
types: 3,205,460,
coercions: 211,579,
joins: 0/1,454}
Result size of Simplifier iteration=2
= {terms: 44,014, types: 3,149,130, coercions: 141,985, joins: 0/6}
Result size of Simplifier
= {terms: 44,006, types: 3,149,102, coercions: 141,942, joins: 0/6}
!!! Simplifier [Distribution.SPDX.LicenseId]: finished in 1395.27 milliseconds, allocated 951.896 megabytes
*** Specialise [Distribution.SPDX.LicenseId]:
Result size of Specialise
= {terms: 44,572,
types: 3,178,208,
coercions: 188,895,
joins: 2/14}
!!! Specialise [Distribution.SPDX.LicenseId]: finished in 324.63 milliseconds, allocated 514.022 megabytes
*** Float out(FOS {Lam = Just 0,
Consts = True,
OverSatApps = False}) [Distribution.SPDX.LicenseId]:
Result size of Float out(FOS {Lam = Just 0,
Consts = True,
OverSatApps = False})
= {terms: 65,000,
types: 6,064,735,
coercions: 188,895,
joins: 1/16}
!!! Float out(FOS {Lam = Just 0,
Consts = True,
OverSatApps = False}) [Distribution.SPDX.LicenseId]: finished in 304.82 milliseconds, allocated 215.492 megabytes
*** Simplifier [Distribution.SPDX.LicenseId]:
Result size of Simplifier iteration=1
= {terms: 80,726,
types: 6,074,463,
coercions: 137,281,
joins: 369/747}
Result size of Simplifier iteration=2
= {terms: 82,411,
types: 6,077,232,
coercions: 137,284,
joins: 362/737}
Result size of Simplifier
= {terms: 81,326,
types: 6,075,424,
coercions: 137,284,
joins: 362/737}
!!! Simplifier [Distribution.SPDX.LicenseId]: finished in 2011.86 milliseconds, allocated 1404.780 megabytes
*** Simplifier [Distribution.SPDX.LicenseId]:
Result size of Simplifier iteration=1
= {terms: 79,402,
types: 6,067,173,
coercions: 137,337,
joins: 365/750}
Result size of Simplifier iteration=2
= {terms: 79,329,
types: 6,067,074,
coercions: 137,309,
joins: 361/741}
Result size of Simplifier iteration=3
= {terms: 79,323,
types: 6,067,072,
coercions: 137,309,
joins: 361/741}
Result size of Simplifier
= {terms: 79,323,
types: 6,067,072,
coercions: 137,309,
joins: 361/741}
!!! Simplifier [Distribution.SPDX.LicenseId]: finished in 2697.31 milliseconds, allocated 1617.643 megabytes
*** Simplifier [Distribution.SPDX.LicenseId]:
Result size of Simplifier iteration=1
= {terms: 79,973,
types: 6,072,111,
coercions: 142,026,
joins: 361/781}
Result size of Simplifier iteration=2
= {terms: 79,867,
types: 6,071,910,
coercions: 142,026,
joins: 364/774}
Result size of Simplifier
= {terms: 79,867,
types: 6,071,910,
coercions: 142,026,
joins: 364/774}
!!! Simplifier [Distribution.SPDX.LicenseId]: finished in 1711.33 milliseconds, allocated 1133.686 megabytes
*** Float inwards [Distribution.SPDX.LicenseId]:
Result size of Float inwards
= {terms: 79,867,
types: 6,071,910,
coercions: 142,026,
joins: 364/774}
!!! Float inwards [Distribution.SPDX.LicenseId]: finished in 118.42 milliseconds, allocated 52.564 megabytes
*** Called arity analysis [Distribution.SPDX.LicenseId]:
Result size of Called arity analysis
= {terms: 79,867,
types: 6,071,910,
coercions: 142,026,
joins: 364/774}
!!! Called arity analysis [Distribution.SPDX.LicenseId]: finished in 473.15 milliseconds, allocated 130.823 megabytes
*** Simplifier [Distribution.SPDX.LicenseId]:
Result size of Simplifier iteration=1
= {terms: 79,867,
types: 6,071,910,
coercions: 142,026,
joins: 364/774}
Result size of Simplifier
= {terms: 79,867,
types: 6,071,910,
coercions: 142,026,
joins: 364/774}
!!! Simplifier [Distribution.SPDX.LicenseId]: finished in 1211.58 milliseconds, allocated 754.988 megabytes
*** Demand analysis [Distribution.SPDX.LicenseId]:
Result size of Demand analysis
= {terms: 79,867,
types: 6,071,910,
coercions: 142,026,
joins: 364/774}
!!! Demand analysis [Distribution.SPDX.LicenseId]: finished in 431.87 milliseconds, allocated 413.455 megabytes
*** Worker Wrapper binds [Distribution.SPDX.LicenseId]:
Result size of Worker Wrapper binds
= {terms: 96,119,
types: 6,107,852,
coercions: 143,834,
joins: 364/4,166}
!!! Worker Wrapper binds [Distribution.SPDX.LicenseId]: finished in 108.26 milliseconds, allocated 29.227 megabytes
*** Simplifier [Distribution.SPDX.LicenseId]:
Result size of Simplifier iteration=1
= {terms: 94,962,
types: 6,107,487,
coercions: 148,001,
joins: 364/1,933}
Result size of Simplifier iteration=2
= {terms: 79,771,
types: 6,072,298,
coercions: 141,864,
joins: 364/763}
Result size of Simplifier
= {terms: 79,769,
types: 6,072,286,
coercions: 141,864,
joins: 364/763}
!!! Simplifier [Distribution.SPDX.LicenseId]: finished in 2572.15 milliseconds, allocated 1520.219 megabytes
*** Exitification transformation [Distribution.SPDX.LicenseId]:
Result size of Exitification transformation
= {terms: 79,769,
types: 6,072,286,
coercions: 141,864,
joins: 364/763}
!!! Exitification transformation [Distribution.SPDX.LicenseId]: finished in 97.67 milliseconds, allocated 20.372 megabytes
*** Float out(FOS {Lam = Just 0,
Consts = True,
OverSatApps = True}) [Distribution.SPDX.LicenseId]:
Result size of Float out(FOS {Lam = Just 0,
Consts = True,
OverSatApps = True})
= {terms: 79,862,
types: 6,072,383,
coercions: 141,864,
joins: 0/397}
!!! Float out(FOS {Lam = Just 0,
Consts = True,
OverSatApps = True}) [Distribution.SPDX.LicenseId]: finished in 609.06 milliseconds, allocated 600.319 megabytes
*** Common sub-expression [Distribution.SPDX.LicenseId]:
Result size of Common sub-expression
= {terms: 78,945,
types: 6,072,251,
coercions: 141,850,
joins: 0/397}
!!! Common sub-expression [Distribution.SPDX.LicenseId]: finished in 174.62 milliseconds, allocated 139.020 megabytes
*** Float inwards [Distribution.SPDX.LicenseId]:
Result size of Float inwards
= {terms: 78,945,
types: 6,072,251,
coercions: 141,850,
joins: 0/397}
!!! Float inwards [Distribution.SPDX.LicenseId]: finished in 201.54 milliseconds, allocated 49.539 megabytes
*** Simplifier [Distribution.SPDX.LicenseId]:
Result size of Simplifier iteration=1
= {terms: 75,571,
types: 6,069,676,
coercions: 141,850,
joins: 0/397}
Result size of Simplifier
= {terms: 75,571,
types: 6,069,676,
coercions: 141,850,
joins: 0/397}
!!! Simplifier [Distribution.SPDX.LicenseId]: finished in 1274.23 milliseconds, allocated 864.666 megabytes
*** Demand analysis [Distribution.SPDX.LicenseId]:
Result size of Demand analysis
= {terms: 75,571,
types: 6,069,676,
coercions: 141,850,
joins: 0/397}
!!! Demand analysis [Distribution.SPDX.LicenseId]: finished in 387.90 milliseconds, allocated 388.882 megabytes
*** CoreTidy [Distribution.SPDX.LicenseId]:
Result size of Tidy Core
= {terms: 75,473,
types: 6,041,371,
coercions: 132,435,
joins: 0/393}
!!! CoreTidy [Distribution.SPDX.LicenseId]: finished in 1596.20 milliseconds, allocated 904.122 megabytes
*** CorePrep [Distribution.SPDX.LicenseId]:
Result size of CorePrep
= {terms: 88,848,
types: 6,105,346,
coercions: 132,435,
joins: 0/1,902}
!!! CorePrep [Distribution.SPDX.LicenseId]: finished in 235.50 milliseconds, allocated 137.218 megabytes
*** Stg2Stg:
*** CodeGen [Distribution.SPDX.LicenseId]:
!!! CodeGen [Distribution.SPDX.LicenseId]: finished in 2583.36 milliseconds, allocated 2989.573 megabytes
*** Assembler:
/nix/store/ghzg4kg0sjif58smj2lfm2bdvjwim85y-gcc-wrapper-7.4.0/bin/cc -fno-stack-protector -DTABLES_NEXT_TO_CODE -iquoteDistribution/SPDX -no-pie -x assembler -c /tmp/ghc14143_0/ghc_16.s -o Distribution/SPDX/LicenseId.o
Upsweep completely successful.
*** Deleting temp files:
Deleting: /tmp/ghc14143_0/ghc_15.s /tmp/ghc14143_0/ghc_16.s /tmp/ghc14143_0/ghc_17.c
Warning: deleting non-existent /tmp/ghc14143_0/ghc_15.s
Warning: deleting non-existent /tmp/ghc14143_0/ghc_17.c
link(batch): upsweep (partially) failed OR
Main.main not exported; not linking.
*** Deleting temp files:
Deleting: /tmp/ghc14143_0/ghc_1.hscpp /tmp/ghc14143_0/ghc_11.hscpp /tmp/ghc14143_0/ghc_13.hscpp /tmp/ghc14143_0/ghc_3.hscpp /tmp/ghc14143_0/ghc_5.hscpp /tmp/ghc14143_0/ghc_7.hscpp /tmp/ghc14143_0/ghc_9.hscpp
*** Deleting temp dirs:
Deleting: /tmp/ghc14143_0
real 0m26.301s
user 0m25.712s
sys 0m0.607s
Edited by Sylvain Henry