-fignore-interface-pragmas doesn't work well with --make
Interfaces files are currently only read once but how they are read depends on -fignore-interface-pragmas. If the flag is
turned on then the files are read without pragma information (unfolding info, rules, demand info and so on). This causes
problems with --make, if a later module is compiled with optimisation and requires the same interface file because it has already been loaded and stripped -- during optimization imported identifiers will not have pragma info and so optimisation decisions will be
different.
There are quite a few tickets reporting this, this ticket tracks progress to fixing them all: #13002, #12847 (closed), #9370, #8635, #20021
Proposed Solution
The proposed solution is to always read pragma information from an interface file and then decide at the use site whether to make use of it or not.
There are the currently serialised pragmas:
data IfaceInfoItem
= HsArity Arity
| HsDmdSig DmdSig
| HsCprSig CprSig
| HsInline InlinePragma
| HsUnfold Bool -- True <=> isStrongLoopBreaker is true
IfaceUnfolding -- See Note [Expose recursive functions]
| HsNoCafRefs
| HsLevity -- Present <=> never representation-polymorphic
| HsLFInfo IfaceLFInfThe other place the flag takes effect is when deserialising IfaceRules.
The current iteration of this/per proposed approach is at !6080. We always read all info from an interface file but then only guard the use sites of HsUnfold and RULES pragmas. This prevents cross-module inlining and cross-module rule firings. Reading the other pragmas seems to have a generally positive effect on compile-time.
Activity
added needs triage label
assigned to @mpickering
On CI I tested a patch which
- Never exposes any unfoldings with -O0
- Never applied any rules
The CI looks like:
Baseline Test Metric value New value Change --------------------------------------------------------------------------------- ManyAlternatives(normal) ghc/alloc 797718856.0 794995792.0 -0.3% ManyConstructors(normal) ghc/alloc 4391739116.0 4365118280.0 -0.6% MultiLayerModules(normal) ghc/alloc 4168212944.0 4165876120.0 -0.1% MultiLayerModulesRecomp(normal) ghc/alloc 879499900.0 879598824.0 +0.0% PmSeriesG(normal) ghc/alloc 51288292.0 74821696.0 +45.9% BAD PmSeriesS(normal) ghc/alloc 64239212.0 61925480.0 -3.6% PmSeriesT(normal) ghc/alloc 94919656.0 94820672.0 -0.1% PmSeriesV(normal) ghc/alloc 63258808.0 61750552.0 -2.4% T10421(normal) ghc/alloc 120241200.0 120090336.0 -0.1% T10421a(normal) ghc/alloc 84205696.0 84090256.0 -0.1% T10547(normal) ghc/alloc 29480240.0 29647472.0 +0.6% T10858(normal) ghc/alloc 188459028.0 188211192.0 -0.1% T11195(normal) ghc/alloc 284331452.0 204360368.0 -28.1% GOOD T11276(normal) ghc/alloc 142685176.0 147972984.0 +3.7% T11303b(normal) ghc/alloc 46666096.0 51201552.0 +9.7% T11374(normal) ghc/alloc 257014800.0 215013008.0 -16.3% GOOD T11545(normal) ghc/alloc 2664620332.0 2664436568.0 -0.0% T11822(normal) ghc/alloc 140207252.0 142471008.0 +1.6% T12150(optasm) ghc/alloc 86999108.0 86840208.0 -0.2% T12227(normal) ghc/alloc 467104728.0 466827056.0 -0.1% T12234(optasm) ghc/alloc 61091540.0 60940232.0 -0.2% T12425(optasm) ghc/alloc 104319740.0 104136016.0 -0.2% T12545(normal) ghc/alloc 1708739016.0 1690183744.0 -1.1% T12707(normal) ghc/alloc 997873708.0 764217936.0 -23.4% GOOD T13035(normal) ghc/alloc 109099932.0 111996704.0 +2.7% BAD T13056(optasm) ghc/alloc 412020984.0 411682536.0 -0.1% T13253(normal) ghc/alloc 356708000.0 356595632.0 -0.0% T13253-spj(normal) ghc/alloc 157712268.0 157550968.0 -0.1% T13379(normal) ghc/alloc 373774352.0 340975368.0 -8.8% GOOD T13701(normal) ghc/alloc 2416297480.0 2412576896.0 -0.2% T13719(normal) ghc/alloc 4388467724.0 4381956032.0 -0.1% T14052(ghci) ghc/alloc 2183112512.0 2275591032.0 +4.2% T14683(normal) ghc/alloc 3066290320.0 3206903656.0 +4.6% BAD T14697(normal) ghc/alloc 366527328.0 366280400.0 -0.1% T15164(normal) ghc/alloc 1378675680.0 1379083648.0 +0.0% T15304(normal) ghc/alloc 1390631792.0 1390714144.0 +0.0% T15630(normal) ghc/alloc 173075512.0 172978088.0 -0.1% T16190(normal) ghc/alloc 282129076.0 282882808.0 +0.3% T16577(normal) ghc/alloc 7872941236.0 7876563144.0 +0.0% T17096(normal) ghc/alloc 298166112.0 262769984.0 -11.9% GOOD T17516(normal) ghc/alloc 1204127052.0 1204218496.0 +0.0% T17836(normal) ghc/alloc 1127591444.0 1252474104.0 +11.1% BAD T17836b(normal) ghc/alloc 54878648.0 63737928.0 +16.1% BAD T17977(normal) ghc/alloc 47699248.0 48921408.0 +2.6% T17977b(normal) ghc/alloc 43067900.0 43545792.0 +1.1% T18140(normal) ghc/alloc 109574144.0 109451928.0 -0.1% T18223(normal) ghc/alloc 5631149868.0 5631282616.0 +0.0% T18282(normal) ghc/alloc 156033608.0 155903064.0 -0.1% T18304(normal) ghc/alloc 97610360.0 97416552.0 -0.2% T18478(normal) ghc/alloc 775495992.0 745709920.0 -3.8% T18698a(normal) ghc/alloc 375253600.0 375117152.0 -0.0% T18698b(normal) ghc/alloc 457587088.0 457449896.0 -0.0% T18923(normal) ghc/alloc 71916444.0 71794016.0 -0.2% T1969(normal) ghc/alloc 808455468.0 899906840.0 +11.3% BAD T3064(normal) ghc/alloc 198399728.0 196187816.0 -1.1% T3294(normal) ghc/alloc 1645321744.0 374930736.0 -77.2% GOOD T4801(normal) ghc/alloc 313851596.0 317811176.0 +1.3% T5030(normal) ghc/alloc 370239316.0 399492184.0 +7.9% BAD T5321FD(normal) ghc/alloc 284485500.0 166820624.0 -41.4% GOOD T5321Fun(normal) ghc/alloc 329129560.0 200307328.0 -39.1% GOOD T5631(normal) ghc/alloc 599759960.0 599081576.0 -0.1% T5642(normal) ghc/alloc 508119292.0 507976464.0 -0.0% T5837(normal) ghc/alloc 38051384.0 38017992.0 -0.1% T6048(optasm) ghc/alloc 87190884.0 87065872.0 -0.1% T783(normal) ghc/alloc 407740264.0 407811664.0 +0.0% T9020(optasm) ghc/alloc 272142256.0 272122968.0 -0.0% T9198(normal) ghc/alloc 49696348.0 90811328.0 +82.7% BAD T9233(normal) ghc/alloc 744083452.0 744002704.0 -0.0% T9630(normal) ghc/alloc 1533904648.0 1533799416.0 -0.0% T9675(optasm) ghc/alloc 466331328.0 466237744.0 -0.0% T9872a(normal) ghc/alloc 1796262024.0 1796678440.0 +0.0% T9872b(normal) ghc/alloc 2310831280.0 2311246376.0 +0.0% T9872c(normal) ghc/alloc 1841330864.0 1841747520.0 +0.0% T9872d(normal) ghc/alloc 556755844.0 563102736.0 +1.1% BAD T9961(normal) ghc/alloc 385476436.0 385395200.0 -0.0% WWRec(normal) ghc/alloc 613541748.0 613674752.0 +0.0% hie002(normal) ghc/alloc 9335398952.0 9339373664.0 +0.0% parsing001(normal) ghc/alloc 510771548.0 510745472.0 -0.0%- T9872d - Program size is bigger because of Typeable data con wrappers not being inlined
- T9198 - A huge type is left in the program because the
adefinition is not inlined.a :: IO a -> (IO () -> t) -> t a m cont = cont (m >> putStrLn "a") - T5030 - Coercions in program approximately 30% more, quite a bit of unfolding not happening of local definitions.
- T1969 - Bigger program size, rules not firing.
Before: Tidy size (terms,types,coercions) T1969: 12701 45510 0 After: Tidy size (terms,types,coercions) T1969: 15704 69520 0 - T17836b - Bigger program size
Before: Tidy size (terms,types,coercions) PM: 146 319 1 After: Tidy size (terms,types,coercions) PM: 258 343 1 - T17836 - Bigger program size (Not inlining RecCons1 pattern synonym matcher)
before: Tidy size (terms,types,coercions) PM: 4105 10003 32 After: Tidy size (terms,types,coercions) PM: 5704 9464 32
Those are the regressions I looked into already. My conclusion is that sometimes inlining can reduce code size, which reduces compile times because the generated code is smaller. I am not sure we should be really optimising for code generation time with -O0, perhaps if most of these tests are supposed to test the typechecker we should actually pass
-fno-codeto them.Let's keep sight of the major payoff here: the ability to read all the IdInfo in an interface file, even with -O0, to avoid the problems with --make or GHCi when we want to see a module both and without its IdInfo. With that in mind, I think that some quite big swings in compile time (some up, some down) may be quite acceptable, esp if its "down" for "typical" cases like compiling Cabal.
Can we try nofib -O0 to see the effect on runtime? Or (probably better) cabal or ghc itself? If, say, -O0 runtime doubled with this change we'd probably think again.
Have you managed to gather any data about the number of interface files read
- With -O0 today
- With -O today
- With "read all pragmas" plus no-inlining and no-rules, in your branch.
@mpickering It's been a few weeks any updates?
removed needs triage label
added Ttask label
added Pnormal label
added driver label
mentioned in merge request !6080
mentioned in issue #17931
mentioned in merge request !4350
changed milestone to %9.8.1
mentioned in commit a0b8918e
mentioned in merge request !9869 (closed)
mentioned in commit a060fa30
mentioned in commit b6cbab39
mentioned in commit b8172308
closed with commit 0ada4547
mentioned in commit 0ada4547
mentioned in commit cbd60c81
mentioned in issue #23913
marked this issue as related to #23913
changed milestone to %9.10.1
changed milestone to %9.10.2
mentioned in issue #24658
assigned to @alt-romes
unassigned @mpickering and @alt-romes
removed milestone %9.10.2
