8.8.1: building these tests: ghc: out of memory
Summary
GHC-8.8.1 fails ghc: out of memory
. The module contains test-templates with a lot of constaints.
I'm not sure that's the problem, but it's the only anormal thing in that module.
https://github.com/isomorphism/these/blob/4ada8f38dcc726657abb97cd69311c9ddff098a2/these-tests/test/Tests/Semialign.hs
semialignLaws
:: forall (f :: * -> *).
( Semialign f, Unzip f, Foldable f, Typeable1 f
, Eq (f A), Show (f A), Arbitrary (f A)
, Eq (f B), Show (f B), Arbitrary (f B)
, Eq (f C), Show (f C), Arbitrary (f C)
, Eq (f (A, (B, C))), Show (f (A, (B, C)))
, Eq (f (A, A)), Show (f (A, A))
, Eq (f (A, B)), Show (f (A, B)), Arbitrary (f (A, B))
, Eq (f (C, Int)), Show (f (C, Int))
, Eq (f (These (A, B) C)), Show (f (These (A, B) C))
, Eq (f (These (A, C) (B, C))), Show (f (These (A, C) (B, C)))
, Eq (f (These A (A, B))), Show (f (These A (A, B)))
, Eq (f (These A (These B C))), Show (f (These A (These B C)))
, Eq (f (These A A)), Show (f (These A A))
, Eq (f (These A B)), Show (f (These A B)), Arbitrary (f (These A B))
, Eq (f (These C Int)), Show (f (These C Int))
)
=> CSemialign f
-> TestTree
Steps to reproduce
An example run is: https://travis-ci.org/isomorphism/these/jobs/540233236
https://github.com/isomorphism/these commit 12fe6b9f1d5832727cdbc56520451bb0a40a8a79
I tried to build with GHC-8.6.5 and 8.4.4 locally. Both clearly spend time compiling Tests.Semialign
module, but eventually complete.
GHC-8.4.4: 484,030,616 bytes maximum residency (24 sample(s))
GHC-8.6.5: 571,116,592 bytes maximum residency (28 sample(s))
ghc-8.8.1 sometimes exits with ghc: out of memory
locally too. after residence is close to 4G. (top shows VIRT 4459M on my machine, so probably then). often it succeeds with:
GHC-8.8.1: 1,478,855,504 bytes maximum residency (23 sample(s))
GHC-8.8.1
Parser [Tests.Semialign]: alloc=17099512 time=13.289
Renamer/typechecker [Tests.Semialign]: alloc=622596360 time=653.357
Desugar [Tests.Semialign]: alloc=42711648 time=39.048
Simplifier [Tests.Semialign]: alloc=113355424 time=279.737
Specialise [Tests.Semialign]: alloc=221084712 time=186.006
Float out(FOS {Lam = Just 0, Consts = True, OverSatApps = False}) [Tests.Semialign]: alloc=402332456 time=392.162
Simplifier [Tests.Semialign]: alloc=13548552280 time=12694.478
Simplifier [Tests.Semialign]: alloc=11332104720 time=10981.635
Simplifier [Tests.Semialign]: alloc=9384085560 time=9783.320
Float inwards [Tests.Semialign]: alloc=6656 time=0.007
Called arity analysis [Tests.Semialign]: alloc=7112 time=0.007
Simplifier [Tests.Semialign]: alloc=10640681528 time=11284.197
Demand analysis [Tests.Semialign]: alloc=1781341416 time=810.240
Worker Wrapper binds [Tests.Semialign]: alloc=55062400 time=54.422
Simplifier [Tests.Semialign]: alloc=10441261416 time=11401.637
Exitification transformation [Tests.Semialign]: alloc=5864 time=0.006
Float out(FOS {Lam = Just 0, Consts = True, OverSatApps = True}) [Tests.Semialign]: alloc=1777108376 time=1821.907
Common sub-expression [Tests.Semialign]: alloc=5872 time=0.006
Float inwards [Tests.Semialign]: alloc=6656 time=0.005
Simplifier [Tests.Semialign]: alloc=9737252680 time=8387.034
Demand analysis [Tests.Semialign]: alloc=1448084904 time=629.879
... crash...
GHC-8.6.5
Parser [Tests.Semialign]: alloc=17115224 time=8.716
Renamer/typechecker [Tests.Semialign]: alloc=515636600 time=353.887
Desugar [Tests.Semialign]: alloc=44228232 time=21.772
Simplifier [Tests.Semialign]: alloc=107580536 time=82.306
Specialise [Tests.Semialign]: alloc=221321520 time=159.225
Float out(FOS {Lam = Just 0, Consts = True, OverSatApps = False}) [Tests.Semialign]: alloc=468220288 time=397.442
Simplifier [Tests.Semialign]: alloc=13214759992 time=11697.655
Simplifier [Tests.Semialign]: alloc=11170412000 time=9980.475
Simplifier [Tests.Semialign]: alloc=9269751072 time=8829.120
Float inwards [Tests.Semialign]: alloc=6600 time=0.007
Called arity analysis [Tests.Semialign]: alloc=7032 time=0.008
Simplifier [Tests.Semialign]: alloc=10709019448 time=9962.258
Demand analysis [Tests.Semialign]: alloc=1848788640 time=2468.301
Worker Wrapper binds [Tests.Semialign]: alloc=53751000 time=43.795
Simplifier [Tests.Semialign]: alloc=10370869448 time=9045.405
Exitification transformation [Tests.Semialign]: alloc=5784 time=0.007
Float out(FOS {Lam = Just 0, Consts = True, OverSatApps = True}) [Tests.Semialign]: alloc=2192354960 time=4228.115
Common sub-expression [Tests.Semialign]: alloc=5792 time=0.006
Float inwards [Tests.Semialign]: alloc=6600 time=0.005
Simplifier [Tests.Semialign]: alloc=7608096824 time=7127.607
Demand analysis [Tests.Semialign]: alloc=1535551672 time=853.090
CoreTidy [Tests.Semialign]: alloc=508247296 time=446.940
CorePrep [Tests.Semialign]: alloc=5088 time=0.028
CodeGen [Tests.Semialign]: alloc=13833595840 time=8867.715
Looks very similar
- GHC-8.8.1 first simplifier uses more time
- GHC-8.6.5 demand analysis use more time
- GHC-8.6.5 stays in 4G residence size and completes.
Trying to minimise
I tried to minimize in test.hs, but there GHC-8.8.1 performs better than 8.6.5 or 8.4.4
If I add {-# OPTIONS_GHC -O0 #-}
to the Tests.Semialign
module, it compiles instantly.
Workaround
I added
{-# NOINLINE semialignLaws' #-}
{-# NOINLINE unzipLaws' #-}
{-# NOINLINE alignLaws' #-}
{-# NOINLINE zipLaws' #-}
{-# NOINLINE unalignLaws' #-}
which made GHC-8.8.1 timings to to (clearly smaller allocations):
Parser [Tests.Semialign]: alloc=17870472 time=10.965
Renamer/typechecker [Tests.Semialign]: alloc=622615824 time=555.566
Desugar [Tests.Semialign]: alloc=42703432 time=28.024
Simplifier [Tests.Semialign]: alloc=113650456 time=220.868
Specialise [Tests.Semialign]: alloc=114298704 time=102.119
Float out(FOS {Lam = Just 0, Consts = True, OverSatApps = False}) [Tests.Semialign]: alloc=213018864 time=153.494
Simplifier [Tests.Semialign]: alloc=3701639120 time=3194.401
Simplifier [Tests.Semialign]: alloc=3082472184 time=3074.771
Simplifier [Tests.Semialign]: alloc=2744323304 time=2195.115
Float inwards [Tests.Semialign]: alloc=6656 time=0.007
Called arity analysis [Tests.Semialign]: alloc=7112 time=0.006
Simplifier [Tests.Semialign]: alloc=3142528080 time=2937.351
Demand analysis [Tests.Semialign]: alloc=424843296 time=210.216
Worker Wrapper binds [Tests.Semialign]: alloc=13230248 time=19.423
Simplifier [Tests.Semialign]: alloc=2298226072 time=2547.377
Exitification transformation [Tests.Semialign]: alloc=5864 time=0.006
Float out(FOS {Lam = Just 0, Consts = True, OverSatApps = True}) [Tests.Semialign]: alloc=469011152 time=445.610
Common sub-expression [Tests.Semialign]: alloc=5872 time=0.005
Float inwards [Tests.Semialign]: alloc=6656 time=0.004
Simplifier [Tests.Semialign]: alloc=2771650824 time=3100.974
Demand analysis [Tests.Semialign]: alloc=315597896 time=122.616
CoreTidy [Tests.Semialign]: alloc=151357712 time=133.801
CorePrep [Tests.Semialign]: alloc=5168 time=0.017
CodeGen [Tests.Semialign]: alloc=3228967624 time=1897.575
Expected behavior
I think GHC should realise not to inline the above "law collections". They are big, they are used more than once.
Environment
- GHC version used: 8.8.0.20190424+git.1.2ffe559c
Optional:
- Operating System: Linux
- System Architecture: amd64