This MR is essentially https://gitlab.haskell.org/ghc/haddock/-/tree/hi-haddock rebased onto https://gitlab.haskell.org/ghc/haddock/-/tree/ghc-head, with small fixes, some formatting/comments, and test fixes.
I've tested performance by generating documentation on the Agda codebase. With these patches, and those in the corresponding GHC patch, the -s statistics look like this:
37,486,323,080 bytes allocated in the heap
10,910,917,392 bytes copied during GC
409,821,536 bytes maximum residency (31 sample(s))
7,524,000 bytes maximum slop
1145 MiB total memory in use (0 MiB lost due to fragmentation)
Tot time (elapsed) Avg pause Max pause
Gen 0 8945 colls, 0 par 5.827s 5.964s 0.0007s 0.0378s
Gen 1 31 colls, 0 par 3.890s 4.029s 0.1300s 0.4070s
TASKS: 5 (1 bound, 4 peak workers (4 total), using -N1)
SPARKS: 2 (0 converted, 0 overflowed, 0 dud, 0 GC'd, 2 fizzled)
INIT time 0.004s ( 0.004s elapsed)
MUT time 9.045s ( 10.824s elapsed)
GC time 9.717s ( 9.993s elapsed)
EXIT time 0.012s ( 0.011s elapsed)
Total time 18.779s ( 20.832s elapsed)
Alloc rate 4,144,346,991 bytes per MUT second
Productivity 48.2% of total user, 52.0% of total elapsedAnd the baseline (default haddock 9.6.1):
40,312,405,088 bytes allocated in the heap
12,933,710,968 bytes copied during GC
1,034,675,264 bytes maximum residency (19 sample(s))
7,060,416 bytes maximum slop
2806 MiB total memory in use (0 MiB lost due to fragmentation)
Tot time (elapsed) Avg pause Max pause
Gen 0 9620 colls, 0 par 5.840s 5.916s 0.0006s 0.0166s
Gen 1 19 colls, 0 par 4.002s 4.236s 0.2230s 0.7593s
TASKS: 5 (1 bound, 4 peak workers (4 total), using -N1)
SPARKS: 0 (0 converted, 0 overflowed, 0 dud, 0 GC'd, 0 fizzled)
INIT time 0.001s ( 0.001s elapsed)
MUT time 9.700s ( 10.783s elapsed)
GC time 9.842s ( 10.153s elapsed)
EXIT time 0.013s ( 0.000s elapsed)
Total time 19.557s ( 20.937s elapsed)
Alloc rate 4,155,729,971 bytes per MUT second
Productivity 49.6% of total user, 51.5% of total elapsedMax residency is halved. This is even more of an improvement over the current ghc-head branch, which produces these statistics:
37,312,388,296 bytes allocated in the heap
11,240,716,688 bytes copied during GC
555,377,456 bytes maximum residency (24 sample(s))
6,524,112 bytes maximum slop
1574 MiB total memory in use (0 MiB lost due to fragmentation)
Tot time (elapsed) Avg pause Max pause
Gen 0 8914 colls, 0 par 6.011s 6.106s 0.0007s 0.0193s
Gen 1 24 colls, 0 par 3.929s 4.148s 0.1728s 0.5674s
TASKS: 5 (1 bound, 4 peak workers (4 total), using -N1)
SPARKS: 2 (0 converted, 0 overflowed, 0 dud, 0 GC'd, 2 fizzled)
INIT time 0.005s ( 0.005s elapsed)
MUT time 9.701s ( 11.074s elapsed)
GC time 9.940s ( 10.255s elapsed)
EXIT time 0.019s ( 0.012s elapsed)
Total time 19.665s ( 21.345s elapsed)
Alloc rate 3,846,058,284 bytes per MUT second
Productivity 49.3% of total user, 51.9% of total elapsed