GHC issueshttps://gitlab.haskell.org/ghc/ghc/-/issues2022-11-12T16:00:03Zhttps://gitlab.haskell.org/ghc/ghc/-/issues/855Improvements to SpecConstr2022-11-12T16:00:03ZSimon Peyton JonesImprovements to SpecConstrThere are a series of possible improvemnts to SpecConstr, described in
the source code. `compiler/specialise/SpecConstr`
- Specialising for constant parameters
- Specialising for lambda parameters
- Two ideas to do with strictness that ...There are a series of possible improvemnts to SpecConstr, described in
the source code. `compiler/specialise/SpecConstr`
- Specialising for constant parameters
- Specialising for lambda parameters
- Two ideas to do with strictness that look more tricky
Some of them look quite straightforward.
<details><summary>Trac metadata</summary>
| Trac field | Value |
| ---------------------- | ------------ |
| Version | 6.4.2 |
| Type | Task |
| TypeOfFailure | OtherFailure |
| Priority | normal |
| Resolution | Unresolved |
| Component | Compiler |
| Test case | |
| Differential revisions | |
| BlockedBy | |
| Related | |
| Blocking | |
| CC | |
| Operating system | Unknown |
| Architecture | Unknown |
</details>
<!-- {"blocked_by":[],"summary":"Improvements to SpecConstr","status":"New","operating_system":"Unknown","component":"Compiler","related":[],"milestone":"","resolution":"Unresolved","owner":{"tag":"Unowned"},"version":"6.4.2","keywords":[],"differentials":[],"test_case":"","architecture":"Unknown","cc":[""],"type":"Task","description":"There are a series of possible improvemnts to SpecConstr, described in \r\nthe source code. {{{compiler/specialise/SpecConstr}}}\r\n\r\n * Specialising for constant parameters\r\n * Specialising for lambda parameters\r\n * Two ideas to do with strictness that look more tricky\r\n\r\nSome of them look quite straightforward.","type_of_failure":"OtherFailure","blocking":[]} -->https://gitlab.haskell.org/ghc/ghc/-/issues/4941SpecConstr generates functions that do not use their arguments2023-11-20T10:27:26ZSimon Peyton JonesSpecConstr generates functions that do not use their argumentsSee also
* #5302
Consider this function:
```
f :: Int -> (Bool,Bool) -> Bool -> Bool
f 0 x y = y
f n (p,q) y = f (n-1) (p,q) q
```
`SpecConstr` does a reasonable job, but ends up with a function like this:
```
T4908a.f_$s$wf =
...See also
* #5302
Consider this function:
```
f :: Int -> (Bool,Bool) -> Bool -> Bool
f 0 x y = y
f n (p,q) y = f (n-1) (p,q) q
```
`SpecConstr` does a reasonable job, but ends up with a function like this:
```
T4908a.f_$s$wf =
\ (sc_sp4 :: GHC.Prim.Int#)
(sc1_sp5 :: GHC.Types.Bool)
(sc2_sp6 :: GHC.Types.Bool)
(sc3_sp7 :: GHC.Types.Bool) ->
case sc_sp4 of ds_Xom {
__DEFAULT ->
T4908a.f_$s$wf (GHC.Prim.-# ds_Xom 1) sc1_sp5 sc2_sp6 sc2_sp6;
0 -> sc3_sp7
}
```
Note that `sc1_sp5` is passed around the loop but never used.
I had a quick go at trying to make `SpecConstr` cleverer, but absence info requires a fixpoint analysis, which the existing `ArgOcc` stuff doesn't do. Nor can we rely on absence analysis from earlier in the compiler, because CSE invalidates it.
A possibility would be to run strictness/absence analysis again after `SpecConstr`, which would pick this up. I'm not sure what other consequences this would have.
So there's an opportunity here, but I'm not sure how much it matters in practice.
<details><summary>Trac metadata</summary>
| Trac field | Value |
| ---------------------- | ------------ |
| Version | 7.0.1 |
| Type | Task |
| TypeOfFailure | OtherFailure |
| Priority | normal |
| Resolution | Unresolved |
| Component | Compiler |
| Test case | |
| Differential revisions | |
| BlockedBy | |
| Related | |
| Blocking | |
| CC | |
| Operating system | |
| Architecture | |
</details>
<!-- {"blocked_by":[],"summary":"SpecConstr generates functions that do not use their arguments","status":"New","operating_system":"","component":"Compiler","related":[],"milestone":"⊥","resolution":"Unresolved","owner":{"tag":"Unowned"},"version":"7.0.1","keywords":[],"differentials":[],"test_case":"","architecture":"","cc":[""],"type":"Task","description":"Consider this function:\r\n{{{\r\nf :: Int -> (Bool,Bool) -> Bool -> Bool\r\nf 0 x y = y\r\nf n (p,q) y = f (n-1) (p,q) q\r\n}}}\r\n`SpecConstr` does a reasonable job, but ends up with a function like this:\r\n{{{\r\nT4908a.f_$s$wf =\r\n \\ (sc_sp4 :: GHC.Prim.Int#)\r\n (sc1_sp5 :: GHC.Types.Bool)\r\n (sc2_sp6 :: GHC.Types.Bool)\r\n (sc3_sp7 :: GHC.Types.Bool) ->\r\n case sc_sp4 of ds_Xom {\r\n __DEFAULT ->\r\n T4908a.f_$s$wf (GHC.Prim.-# ds_Xom 1) sc1_sp5 sc2_sp6 sc2_sp6;\r\n 0 -> sc3_sp7\r\n }\r\n}}}\r\nNote that `sc1_sp5` is passed around the loop but never used.\r\n\r\nI had a quick go at trying to make `SpecConstr` cleverer, but absence info requires a fixpoint analysis, which the existing `ArgOcc` stuff doesn't do. Nor can we rely on absence analysis from earlier in the compiler, because CSE invalidates it. \r\n\r\nA possibility would be to run strictness/absence analysis again after `SpecConstr`, which would pick this up. I'm not sure what other consequences this would have.\r\n\r\nSo there's an opportunity here, but I'm not sure how much it matters in practice.","type_of_failure":"OtherFailure","blocking":[]} -->⊥https://gitlab.haskell.org/ghc/ghc/-/issues/10346Cross-module SpecConstr2022-05-12T20:34:01ZSimon Peyton JonesCross-module SpecConstrType-class specialisation now happens flawlessly across modules. That is, if I define
```
module DefineF where
f :: Num a => a -> a
{-# INLINEABLE f #-}
f x = ...f x'....
```
then modules that import `DefineF` and call `f` at ...Type-class specialisation now happens flawlessly across modules. That is, if I define
```
module DefineF where
f :: Num a => a -> a
{-# INLINEABLE f #-}
f x = ...f x'....
```
then modules that import `DefineF` and call `f` at some particular type (say `Int`) will generate a specialised copy of `f`'s code.
But this does not happen for `SpecConstr`; we only specialise a function for calls made in the same module. For example:
```
module M where
{-# INLINABLE foo #-}
foo True y = y
foo False (a,b) = foo True (a+b,b)
module X where
import M
bar = ...(foo (x,y))...
```
Here `foo` is called with an explicit `(x,y)` argument in module `X`, and we'd like to !SpecConstr it, as it would be if the call was in module `M`.
All the infrastructure is in place to allow cross-module `SpecConstr`; it just hasn't been done yet. This ticket is to record the idea.8.6.1https://gitlab.haskell.org/ghc/ghc/-/issues/13014Seemingly unnecessary marking of a SpecConstr specialization as a loopbreaker2019-07-07T18:24:06ZnfrisbySeemingly unnecessary marking of a SpecConstr specialization as a loopbreaker!SpecConstr creates the following rules, with the right cajoling. (I've used unboxed integers merely to avoid w/w, which only adds noise in this example.)
(See [ticket:13014\#comment:129293](https://gitlab.haskell.org//ghc/ghc/issues/13...!SpecConstr creates the following rules, with the right cajoling. (I've used unboxed integers merely to avoid w/w, which only adds noise in this example.)
(See [ticket:13014\#comment:129293](https://gitlab.haskell.org//ghc/ghc/issues/13014#note_129293) for a Minimal Working Example.)
```hs
data VL :: [k] -> * where
VLZ :: VL '[]
VLS :: VL as -> VL (a ': as)
lengthVL :: GHC.Types.SPEC -> VL as -> Int#
{-# INLINABLE lengthVL #-}
lengthVL !sPEC VLZ = 0#
lengthVL !sPEC (VLS vl) = 1# +# lengthVL sPEC vl
==================== Tidy Core rules ====================
"SC:lengthVL0" [ALWAYS]
forall (@ a) (@ (as :: [*])) (sc :: VL as).
lengthVL @ (a : as)
SPEC
(VLS
@ * @ (a : as) @ as @ a @~ (<a : as>_N :: (a : as) ~ (a : as)) sc)
= lengthVL_$slengthVL1 @ a @ as sc
"SC:lengthVL1" [ALWAYS]
forall (sc :: VL '[]).
lengthVL @ '[] SPEC sc
= lengthVL_$slengthVL sc
```
But the cons-case specialization, `lengthVL_$slengthVL1`, is marked as a loopbreaker. Consider the following idiomatic usage to see why that is problematic.
```hs
class KnownSpine (as :: [k]) where sing :: VL as
instance KnownSpine '[] where -- '
{-# INLINE sing #-}
sing = VLZ
instance KnownSpine as => KnownSpine (a ': as) where -- '
{-# INLINE sing #-}
sing = VLS sing
example :: Int
example = I# $ lengthVL SPEC (sing :: VL '[Int,Char,Bool])
```
The right-hand side of `example` would ideally be simplified to `3`. It's not, ultimately because the specialization is marked as a loopbreaker.
I switched on `-dverbose-core2core` to track the simplification of the right-hand side of `example`. 1) The `sing` dictionary is unfolded to constructor applications. 2) Those are floated out but then pre-inlined-unconditionally right back in before CSE gets a chance to spoil it. 3) Thus the VLS rule fires. But it only fires once, because of the loopbreaker designation!
I have not yet investigated why the specialization in the cons-case is marked a loopbreaker.
(Even if the specialization wasn't being considered a loopbreaker --- which immediately makes this approach to optimization a dead-end --- I don't know with any certainty how to force the specialization to be inlined in those cases where its right-hand side was relatively large.)https://gitlab.haskell.org/ghc/ghc/-/issues/13535vector test suite uses excessive memory on GHC 8.22023-06-02T11:06:30ZRyan Scottvector test suite uses excessive memory on GHC 8.2First noticed [here](https://github.com/haskell/vector/pull/161#issuecomment-292031845). I haven't managed to boil this down to a test case with no dependencies yet, so for the time being, this requires `vector`. To reproduce, follow the...First noticed [here](https://github.com/haskell/vector/pull/161#issuecomment-292031845). I haven't managed to boil this down to a test case with no dependencies yet, so for the time being, this requires `vector`. To reproduce, follow these steps:
```
$ git clone https://github.com/erikd/vector
$ cd vector/
$ cabal install --only-dependencies --enable-tests -w /opt/ghc/8.2.1/bin/ghc
$ cabal configure --enable-tests -w /opt/ghc/8.2.1/bin/ghc
$ cabal test
```
When building `vector-tests-O2`, GHC will stall when compiling the `Tests.Vector` module. On machines with modest memory allowances (e.g., [the machines used on Travis CI](https://travis-ci.org/haskell/vector/jobs/218749281#L1270)), GHC will be killed with an out-of-memory error after trying to compile `Tests.Vector` for a while.
<details><summary>Trac metadata</summary>
| Trac field | Value |
| ---------------------- | ------------ |
| Version | 8.1 |
| Type | Bug |
| TypeOfFailure | OtherFailure |
| Priority | highest |
| Resolution | Unresolved |
| Component | Compiler |
| Test case | |
| Differential revisions | |
| BlockedBy | |
| Related | |
| Blocking | |
| CC | |
| Operating system | |
| Architecture | |
</details>
<!-- {"blocked_by":[],"summary":"vector test suite uses excessive memory on GHC 8.2","status":"New","operating_system":"","component":"Compiler","related":[],"milestone":"8.2.1","resolution":"Unresolved","owner":{"tag":"Unowned"},"version":"8.1","keywords":[],"differentials":[],"test_case":"","architecture":"","cc":[""],"type":"Bug","description":"First noticed [https://github.com/haskell/vector/pull/161#issuecomment-292031845 here]. I haven't managed to boil this down to a test case with no dependencies yet, so for the time being, this requires `vector`. To reproduce, follow these steps:\r\n\r\n{{{\r\n$ git clone https://github.com/erikd/vector\r\n$ cd vector/\r\n$ cabal install --only-dependencies --enable-tests -w /opt/ghc/8.2.1/bin/ghc\r\n$ cabal configure --enable-tests -w /opt/ghc/8.2.1/bin/ghc\r\n$ cabal test\r\n}}}\r\n\r\nWhen building `vector-tests-O2`, GHC will stall when compiling the `Tests.Vector` module. On machines with modest memory allowances (e.g., [https://travis-ci.org/haskell/vector/jobs/218749281#L1270 the machines used on Travis CI]), GHC will be killed with an out-of-memory error after trying to compile `Tests.Vector` for a while.","type_of_failure":"OtherFailure","blocking":[]} -->⊥Ben GamariBen Gamarihttps://gitlab.haskell.org/ghc/ghc/-/issues/13681Remove deprecated ForceSpecConstr2021-02-10T03:12:27ZMatthew PickeringRemove deprecated ForceSpecConstrThere is a note at the top of `SpecConstr` which says
```
ToDo [Oct 2013]
~~~~~~~~~~~~~~~
1....There is a note at the top of `SpecConstr` which says
```
ToDo [Oct 2013]
~~~~~~~~~~~~~~~
1. Nuke ForceSpecConstr for good (it is subsumed by GHC.Types.SPEC in ghc-prim)
2. Nuke NoSpecConstr
```
`ForceSpecConstr` still exists so this ticket is to track the progress made on removing it.
It isn't clear to me why it is ok to get rid of `NoSpecConstr` (which prevents SpecConstr firing) either.
<details><summary>Trac metadata</summary>
| Trac field | Value |
| ---------------------- | ------------ |
| Version | 8.0.1 |
| Type | Task |
| TypeOfFailure | OtherFailure |
| Priority | normal |
| Resolution | Unresolved |
| Component | Compiler |
| Test case | |
| Differential revisions | |
| BlockedBy | |
| Related | |
| Blocking | |
| CC | |
| Operating system | |
| Architecture | |
</details>
<!-- {"blocked_by":[],"summary":"Remove deprecated ForceSpecConstr","status":"New","operating_system":"","component":"Compiler","related":[],"milestone":"","resolution":"Unresolved","owner":{"tag":"Unowned"},"version":"8.0.1","keywords":["SpecConstr"],"differentials":[],"test_case":"","architecture":"","cc":[""],"type":"Task","description":"There is a note at the top of `SpecConstr` which says \r\n\r\n{{{\r\n ToDo [Oct 2013] \r\n ~~~~~~~~~~~~~~~ \r\n 1. Nuke ForceSpecConstr for good (it is subsumed by GHC.Types.SPEC in ghc-prim) \r\n 2. Nuke NoSpecConstr \r\n}}}\r\n\r\n`ForceSpecConstr` still exists so this ticket is to track the progress made on removing it. \r\n\r\nIt isn't clear to me why it is ok to get rid of `NoSpecConstr` (which prevents SpecConstr firing) either.","type_of_failure":"OtherFailure","blocking":[]} -->https://gitlab.haskell.org/ghc/ghc/-/issues/13694CSE runs before SpecConstr2019-07-07T18:20:34ZMatthew PickeringCSE runs before SpecConstrSpecConstr can lead to more CSE opportunities. Currently CSE is run only once after the full laziness transformation but then not again later in the pipeline. Running it again after the final clean-up simplification might lead to smaller...SpecConstr can lead to more CSE opportunities. Currently CSE is run only once after the full laziness transformation but then not again later in the pipeline. Running it again after the final clean-up simplification might lead to smaller programs.
One example with `-fspec-constr-keen`.
```
module Foo where
main :: [Int]
main = drop 1 [1,2]
mainMODULE :: [Int]
mainMODULE = mydrop 1 [1,2]
mydrop :: Int -> [a] -> [a]
mydrop 0 xs = xs
mydrop n (x:xs) = mydrop (n-1) xs
```
<details><summary>Trac metadata</summary>
| Trac field | Value |
| ---------------------- | ------------ |
| Version | 8.0.1 |
| Type | Bug |
| TypeOfFailure | OtherFailure |
| Priority | normal |
| Resolution | Unresolved |
| Component | Compiler |
| Test case | |
| Differential revisions | |
| BlockedBy | |
| Related | |
| Blocking | |
| CC | |
| Operating system | |
| Architecture | |
</details>
<!-- {"blocked_by":[],"summary":"CSE runs before SpecConstr","status":"New","operating_system":"","component":"Compiler","related":[],"milestone":"","resolution":"Unresolved","owner":{"tag":"Unowned"},"version":"8.0.1","keywords":["SpecConstr,","cse"],"differentials":[],"test_case":"","architecture":"","cc":[""],"type":"Bug","description":"SpecConstr can lead to more CSE opportunities. Currently CSE is run only once after the full laziness transformation but then not again later in the pipeline. Running it again after the final clean-up simplification might lead to smaller programs.\r\n\r\nOne example with `-fspec-constr-keen`.\r\n\r\n{{{\r\nmodule Foo where\r\n\r\nmain :: [Int]\r\nmain = drop 1 [1,2]\r\n\r\nmainMODULE :: [Int]\r\nmainMODULE = mydrop 1 [1,2]\r\n\r\nmydrop :: Int -> [a] -> [a]\r\nmydrop 0 xs = xs\r\nmydrop n (x:xs) = mydrop (n-1) xs\r\n}}}","type_of_failure":"OtherFailure","blocking":[]} -->https://gitlab.haskell.org/ghc/ghc/-/issues/14844SpecConstr also non-recursive top-level functions2022-10-17T16:48:21ZJoachim Breitnermail@joachim-breitner.deSpecConstr also non-recursive top-level functionsIn order to fix the regressions introduced by loopification (#14068), we probably need to get !SpecConstr also do something about non-recursive functions.
In a first iteration, we might want to enable it for non-recursive functions stra...In order to fix the regressions introduced by loopification (#14068), we probably need to get !SpecConstr also do something about non-recursive functions.
In a first iteration, we might want to enable it for non-recursive functions straight-away, and see if it fixes the regressions introduced by Loopification.
But if this turns out to be too slow/too expensive, we should try specializing non-recursive *local* functions *if there is only one call-patterns*. That ought to be a straight win in all cases anyways.
<details><summary>Trac metadata</summary>
| Trac field | Value |
| ---------------------- | ------------ |
| Version | 8.5 |
| Type | Task |
| TypeOfFailure | OtherFailure |
| Priority | normal |
| Resolution | Unresolved |
| Component | Compiler |
| Test case | |
| Differential revisions | |
| BlockedBy | |
| Related | |
| Blocking | |
| CC | |
| Operating system | |
| Architecture | |
</details>
<!-- {"blocked_by":[],"summary":"SpecConstr also non-recursive function","status":"New","operating_system":"","component":"Compiler","related":[],"milestone":"","resolution":"Unresolved","owner":{"tag":"Unowned"},"version":"8.5","keywords":[],"differentials":[],"test_case":"","architecture":"","cc":[""],"type":"Task","description":"In order to fix the regressions introduced by loopification (#14068), we probably need to get !SpecConstr also do something about non-recursive functions.\r\n\r\nIn a first iteration, we might want to enable it for non-recursive functions straight-away, and see if it fixes the regressions introduced by Loopification.\r\n\r\nBut if this turns out to be too slow/too expensive, we should try specializing non-recursive ''local'' functions ''if there is only one call-patterns''. That ought to be a straight win in all cases anyways.\r\n\r\n\r\n\r\n","type_of_failure":"OtherFailure","blocking":[]} -->https://gitlab.haskell.org/ghc/ghc/-/issues/14951SpecConstr needs two runs when one should suffice2023-02-07T11:40:58ZJoachim Breitnermail@joachim-breitner.deSpecConstr needs two runs when one should sufficeThis is a spin-off of #14844, which is a spin-off of #14068, but applies on its own.
Consider this code:
```
module T14844Example (topLvl) where
topLvl large = (bar1, bar2, foo)
where
foo :: Integer -> (a -> b -> Bool) -> (a,b) ...This is a spin-off of #14844, which is a spin-off of #14068, but applies on its own.
Consider this code:
```
module T14844Example (topLvl) where
topLvl large = (bar1, bar2, foo)
where
foo :: Integer -> (a -> b -> Bool) -> (a,b) -> Bool
foo 0 _ _ = False
foo s f t = l s' t
where
l 0 t = False
l 1 t = case t of (x,y) -> f x y
l n (x,y) = l (n-1) (x,y)
s' = large s
bar1 :: Integer -> (a -> b -> Bool) -> a -> b -> Bool
bar1 s f x y = foo s f (x,y)
bar2 :: Integer -> (a -> b -> Bool) -> a -> b -> Bool
bar2 s f x y = foo (s + 1) f (x,y)
```
Status quo: `l` gets specialized, because of the two call patterns
- `l s' t` and
- `l (n-1) (x,y)`
The second one is interesting \*and\* its second argument gets scrutinized (the `scu_occs` field reports `ScrutOcc` for `t`). But `foo` does not get specialized: It does have an interesting call pattern, but `scu_occs` reports `UnkOcc`, because `foo`’s parameters are just passed to `t`.
When we decide to !SpecConstr `l`, we know that one of the calls to `l` is of the shape `s' t0`. This is a boring call, and we do not create a specialization for it. But we create a specialization for `l` using the the other call pattern. This means we know that it would be beneficial if `t0` were a constructor. So can we, at this point, decide to include `t0 ↦ ScrutOcc` in `scu_occs`?
First experiments look good, so I am working on this.https://gitlab.haskell.org/ghc/ghc/-/issues/15069Missed SpecConstr opportunity2020-01-23T19:20:58ZSimon Peyton JonesMissed SpecConstr opportunityIf you look at the final Core for `Csg.calc` in `nofib/real/fulsom` you'll see missed opportunities for `SpecConstr`.
It has nested join points which look like this:
```
join j (x :: Double) = Board (case x of { D# y -> ... })
...If you look at the final Core for `Csg.calc` in `nofib/real/fulsom` you'll see missed opportunities for `SpecConstr`.
It has nested join points which look like this:
```
join j (x :: Double) = Board (case x of { D# y -> ... })
...
in ....(j (D# v))....
```
So the call to $j$ ends up with an explicit `(D# v)` argument, and `SpecConstr`
should catch it. But alas it does not because this happends nestedly, and
`SpecConstr` is worried about exponential blowup.
<details><summary>Trac metadata</summary>
| Trac field | Value |
| ---------------------- | ------------ |
| Version | 8.2.2 |
| Type | Bug |
| TypeOfFailure | OtherFailure |
| Priority | normal |
| Resolution | Unresolved |
| Component | Compiler |
| Test case | |
| Differential revisions | |
| BlockedBy | |
| Related | |
| Blocking | |
| CC | |
| Operating system | |
| Architecture | |
</details>
<!-- {"blocked_by":[],"summary":"Missed SpecConstr opportunity","status":"New","operating_system":"","component":"Compiler","related":[],"milestone":"8.6.1","resolution":"Unresolved","owner":{"tag":"Unowned"},"version":"8.2.2","keywords":["SpecConstr"],"differentials":[],"test_case":"","architecture":"","cc":[""],"type":"Bug","description":"If you look at the final Core for `Csg.calc` in `nofib/real/fulsom` you'll see missed opportunities for `SpecConstr`.\r\n\r\nIt has nested join points which look like this:\r\n{{{\r\n join j (x :: Double) = Board (case x of { D# y -> ... })\r\n ...\r\n in ....(j (D# v))....\r\n}}}\r\nSo the call to $j$ ends up with an explicit `(D# v)` argument, and `SpecConstr`\r\nshould catch it. But alas it does not because this happends nestedly, and\r\n`SpecConstr` is worried about exponential blowup.","type_of_failure":"OtherFailure","blocking":[]} -->https://gitlab.haskell.org/ghc/ghc/-/issues/16017ghc-8.6.1 and ghc-8.6.2 use a lot of memory2019-08-27T08:57:19ZJohn Kyghc-8.6.1 and ghc-8.6.2 use a lot of memoryCurrently GHC uses a lot of memory to build a relatively small module and causes my CI to fail due to there being a 4G memory limit.
The source code can be found here:
https://github.com/haskell-works/hw-json/tree/73368cee21dc72eedd529...Currently GHC uses a lot of memory to build a relatively small module and causes my CI to fail due to there being a 4G memory limit.
The source code can be found here:
https://github.com/haskell-works/hw-json/tree/73368cee21dc72eedd5291ba689f9abf10e7fcd2
The problem module is here:
https://github.com/haskell-works/hw-json/blob/73368cee21dc72eedd5291ba689f9abf10e7fcd2/test/HaskellWorks/Data/Json/Backend/Standard/Succinct/CursorSpec.hs
The build output follows:
```
cabal new-build --enable-tests --enable-benchmarks --project-file="cabal.project" -j${CABAL_THREADS:-4} all
Build profile: -w ghc-8.6.2 -O2
In order, the following will be built (use -v for more details):
- hw-json-0.9.0.1 (lib) (first run)
- hw-json-0.9.0.1 (test:hw-json-test) (first run)
- hw-json-0.9.0.1 (exe:hw-json) (first run)
- hw-json-0.9.0.1 (bench:bench) (first run)
Configuring library for hw-json-0.9.0.1..
Preprocessing library for hw-json-0.9.0.1..
Building library for hw-json-0.9.0.1..
[ 1 of 32] Compiling HaskellWorks.Data.Json.DecodeError ( src/HaskellWorks/Data/Json/DecodeError.hs, /root/project/dist-newstyle/build/x86_64-linux/ghc-8.6.2/hw-json-0.9.0.1/opt/build/HaskellWorks/Data/Json/DecodeError.o )
[ 2 of 32] Compiling HaskellWorks.Data.Json.Internal.Backend.Standard.MakeIndex ( src/HaskellWorks/Data/Json/Internal/Backend/Standard/MakeIndex.hs, /root/project/dist-newstyle/build/x86_64-linux/ghc-8.6.2/hw-json-0.9.0.1/opt/build/HaskellWorks/Data/Json/Internal/Backend/Standard/MakeIndex.o )
[ 3 of 32] Compiling HaskellWorks.Data.Json.Internal.Index ( src/HaskellWorks/Data/Json/Internal/Index.hs, /root/project/dist-newstyle/build/x86_64-linux/ghc-8.6.2/hw-json-0.9.0.1/opt/build/HaskellWorks/Data/Json/Internal/Index.o )
[ 4 of 32] Compiling HaskellWorks.Data.Json.Internal.PartialIndex ( src/HaskellWorks/Data/Json/Internal/PartialIndex.hs, /root/project/dist-newstyle/build/x86_64-linux/ghc-8.6.2/hw-json-0.9.0.1/opt/build/HaskellWorks/Data/Json/Internal/PartialIndex.o )
[ 5 of 32] Compiling HaskellWorks.Data.Json.Internal.Token.Types ( src/HaskellWorks/Data/Json/Internal/Token/Types.hs, /root/project/dist-newstyle/build/x86_64-linux/ghc-8.6.2/hw-json-0.9.0.1/opt/build/HaskellWorks/Data/Json/Internal/Token/Types.o )
[ 6 of 32] Compiling HaskellWorks.Data.Json.Internal.Backend.Standard.Token.Tokenize ( src/HaskellWorks/Data/Json/Internal/Backend/Standard/Token/Tokenize.hs, /root/project/dist-newstyle/build/x86_64-linux/ghc-8.6.2/hw-json-0.9.0.1/opt/build/HaskellWorks/Data/Json/Internal/Backend/Standard/Token/Tokenize.o )
[ 7 of 32] Compiling HaskellWorks.Data.Json.Internal.Token ( src/HaskellWorks/Data/Json/Internal/Token.hs, /root/project/dist-newstyle/build/x86_64-linux/ghc-8.6.2/hw-json-0.9.0.1/opt/build/HaskellWorks/Data/Json/Internal/Token.o )
[ 8 of 32] Compiling HaskellWorks.Data.Json.Internal.Value ( src/HaskellWorks/Data/Json/Internal/Value.hs, /root/project/dist-newstyle/build/x86_64-linux/ghc-8.6.2/hw-json-0.9.0.1/opt/build/HaskellWorks/Data/Json/Internal/Value.o )
[ 9 of 32] Compiling HaskellWorks.Data.Json.Internal.Word8 ( src/HaskellWorks/Data/Json/Internal/Word8.hs, /root/project/dist-newstyle/build/x86_64-linux/ghc-8.6.2/hw-json-0.9.0.1/opt/build/HaskellWorks/Data/Json/Internal/Word8.o )
[10 of 32] Compiling HaskellWorks.Data.Json.Internal.Word64 ( src/HaskellWorks/Data/Json/Internal/Word64.hs, /root/project/dist-newstyle/build/x86_64-linux/ghc-8.6.2/hw-json-0.9.0.1/opt/build/HaskellWorks/Data/Json/Internal/Word64.o )
[11 of 32] Compiling HaskellWorks.Data.Json.Internal.CharLike ( src/HaskellWorks/Data/Json/Internal/CharLike.hs, /root/project/dist-newstyle/build/x86_64-linux/ghc-8.6.2/hw-json-0.9.0.1/opt/build/HaskellWorks/Data/Json/Internal/CharLike.o )
[12 of 32] Compiling HaskellWorks.Data.Json.Internal.Backend.Standard.Blank ( src/HaskellWorks/Data/Json/Internal/Backend/Standard/Blank.hs, /root/project/dist-newstyle/build/x86_64-linux/ghc-8.6.2/hw-json-0.9.0.1/opt/build/HaskellWorks/Data/Json/Internal/Backend/Standard/Blank.o )
[13 of 32] Compiling HaskellWorks.Data.Json.Internal.Backend.Standard.BlankedJson ( src/HaskellWorks/Data/Json/Internal/Backend/Standard/BlankedJson.hs, /root/project/dist-newstyle/build/x86_64-linux/ghc-8.6.2/hw-json-0.9.0.1/opt/build/HaskellWorks/Data/Json/Internal/Backend/Standard/BlankedJson.o )
[14 of 32] Compiling HaskellWorks.Data.Json.Internal.Backend.Standard.ToInterestBits64 ( src/HaskellWorks/Data/Json/Internal/Backend/Standard/ToInterestBits64.hs, /root/project/dist-newstyle/build/x86_64-linux/ghc-8.6.2/hw-json-0.9.0.1/opt/build/HaskellWorks/Data/Json/Internal/Backend/Standard/ToInterestBits64.o )
[15 of 32] Compiling HaskellWorks.Data.Json.Internal.Backend.Standard.ToBalancedParens64 ( src/HaskellWorks/Data/Json/Internal/Backend/Standard/ToBalancedParens64.hs, /root/project/dist-newstyle/build/x86_64-linux/ghc-8.6.2/hw-json-0.9.0.1/opt/build/HaskellWorks/Data/Json/Internal/Backend/Standard/ToBalancedParens64.o )
[16 of 32] Compiling HaskellWorks.Data.Json.Internal.Backend.Standard.IbBp ( src/HaskellWorks/Data/Json/Internal/Backend/Standard/IbBp.hs, /root/project/dist-newstyle/build/x86_64-linux/ghc-8.6.2/hw-json-0.9.0.1/opt/build/HaskellWorks/Data/Json/Internal/Backend/Standard/IbBp.o )
[17 of 32] Compiling HaskellWorks.Data.Json.Backend.Standard.SemiIndex ( src/HaskellWorks/Data/Json/Backend/Standard/SemiIndex.hs, /root/project/dist-newstyle/build/x86_64-linux/ghc-8.6.2/hw-json-0.9.0.1/opt/build/HaskellWorks/Data/Json/Backend/Standard/SemiIndex.o )
[18 of 32] Compiling HaskellWorks.Data.Json.Backend.Simple.SemiIndex ( src/HaskellWorks/Data/Json/Backend/Simple/SemiIndex.hs, /root/project/dist-newstyle/build/x86_64-linux/ghc-8.6.2/hw-json-0.9.0.1/opt/build/HaskellWorks/Data/Json/Backend/Simple/SemiIndex.o )
[19 of 32] Compiling HaskellWorks.Data.Json.LightJson ( src/HaskellWorks/Data/Json/LightJson.hs, /root/project/dist-newstyle/build/x86_64-linux/ghc-8.6.2/hw-json-0.9.0.1/opt/build/HaskellWorks/Data/Json/LightJson.o )
[20 of 32] Compiling HaskellWorks.Data.Json.PartialValue ( src/HaskellWorks/Data/Json/PartialValue.hs, /root/project/dist-newstyle/build/x86_64-linux/ghc-8.6.2/hw-json-0.9.0.1/opt/build/HaskellWorks/Data/Json/PartialValue.o )
[21 of 32] Compiling HaskellWorks.Data.Json.Type ( src/HaskellWorks/Data/Json/Type.hs, /root/project/dist-newstyle/build/x86_64-linux/ghc-8.6.2/hw-json-0.9.0.1/opt/build/HaskellWorks/Data/Json/Type.o )
[22 of 32] Compiling HaskellWorks.Data.Json.Backend.Standard.Cursor ( src/HaskellWorks/Data/Json/Backend/Standard/Cursor.hs, /root/project/dist-newstyle/build/x86_64-linux/ghc-8.6.2/hw-json-0.9.0.1/opt/build/HaskellWorks/Data/Json/Backend/Standard/Cursor.o )
[23 of 32] Compiling HaskellWorks.Data.Json.Cursor ( src/HaskellWorks/Data/Json/Cursor.hs, /root/project/dist-newstyle/build/x86_64-linux/ghc-8.6.2/hw-json-0.9.0.1/opt/build/HaskellWorks/Data/Json/Cursor.o )
[24 of 32] Compiling HaskellWorks.Data.Json.Internal.Backend.Standard.Cursor.Token ( src/HaskellWorks/Data/Json/Internal/Backend/Standard/Cursor/Token.hs, /root/project/dist-newstyle/build/x86_64-linux/ghc-8.6.2/hw-json-0.9.0.1/opt/build/HaskellWorks/Data/Json/Internal/Backend/Standard/Cursor/Token.o )
[25 of 32] Compiling HaskellWorks.Data.Json.Value ( src/HaskellWorks/Data/Json/Value.hs, /root/project/dist-newstyle/build/x86_64-linux/ghc-8.6.2/hw-json-0.9.0.1/opt/build/HaskellWorks/Data/Json/Value.o )
[26 of 32] Compiling HaskellWorks.Data.Json.FromValue ( src/HaskellWorks/Data/Json/FromValue.hs, /root/project/dist-newstyle/build/x86_64-linux/ghc-8.6.2/hw-json-0.9.0.1/opt/build/HaskellWorks/Data/Json/FromValue.o )
[27 of 32] Compiling HaskellWorks.Data.Json.Backend.Standard.LoadCursor ( src/HaskellWorks/Data/Json/Backend/Standard/LoadCursor.hs, /root/project/dist-newstyle/build/x86_64-linux/ghc-8.6.2/hw-json-0.9.0.1/opt/build/HaskellWorks/Data/Json/Backend/Standard/LoadCursor.o )
[28 of 32] Compiling HaskellWorks.Data.Json.LoadCursor ( src/HaskellWorks/Data/Json/LoadCursor.hs, /root/project/dist-newstyle/build/x86_64-linux/ghc-8.6.2/hw-json-0.9.0.1/opt/build/HaskellWorks/Data/Json/LoadCursor.o )
[29 of 32] Compiling HaskellWorks.Data.Json.Backend.Standard.Load ( src/HaskellWorks/Data/Json/Backend/Standard/Load.hs, /root/project/dist-newstyle/build/x86_64-linux/ghc-8.6.2/hw-json-0.9.0.1/opt/build/HaskellWorks/Data/Json/Backend/Standard/Load.o )
[30 of 32] Compiling HaskellWorks.Data.Json.Load ( src/HaskellWorks/Data/Json/Load.hs, /root/project/dist-newstyle/build/x86_64-linux/ghc-8.6.2/hw-json-0.9.0.1/opt/build/HaskellWorks/Data/Json/Load.o )
[31 of 32] Compiling HaskellWorks.Data.Json ( src/HaskellWorks/Data/Json.hs, /root/project/dist-newstyle/build/x86_64-linux/ghc-8.6.2/hw-json-0.9.0.1/opt/build/HaskellWorks/Data/Json.o )
[32 of 32] Compiling Paths_hw_json ( /root/project/dist-newstyle/build/x86_64-linux/ghc-8.6.2/hw-json-0.9.0.1/opt/build/autogen/Paths_hw_json.hs, /root/project/dist-newstyle/build/x86_64-linux/ghc-8.6.2/hw-json-0.9.0.1/opt/build/Paths_hw_json.o )
Configuring test suite 'hw-json-test' for hw-json-0.9.0.1..
Configuring benchmark 'bench' for hw-json-0.9.0.1..
Configuring executable 'hw-json' for hw-json-0.9.0.1..
Preprocessing test suite 'hw-json-test' for hw-json-0.9.0.1..
Building test suite 'hw-json-test' for hw-json-0.9.0.1..
Preprocessing executable 'hw-json' for hw-json-0.9.0.1..
Preprocessing benchmark 'bench' for hw-json-0.9.0.1..
Building benchmark 'bench' for hw-json-0.9.0.1..
Building executable 'hw-json' for hw-json-0.9.0.1..
[ 1 of 11] Compiling HaskellWorks.Data.Json.Backend.Standard.Succinct.Cursor.InterestBitsSpec ( test/HaskellWorks/Data/Json/Backend/Standard/Succinct/Cursor/InterestBitsSpec.hs, /root/project/dist-newstyle/build/x86_64-linux/ghc-8.6.2/hw-json-0.9.0.1/t/hw-json-test/opt/build/hw-json-test/hw-json-test-tmp/HaskellWorks/Data/Json/Backend/Standard/Succinct/Cursor/InterestBitsSpec.o )
[1 of 2] Compiling Main ( bench/Main.hs, /root/project/dist-newstyle/build/x86_64-linux/ghc-8.6.2/hw-json-0.9.0.1/b/bench/opt/build/bench/bench-tmp/Main.o )
[1 of 7] Compiling App.Commands.Types ( app/App/Commands/Types.hs, /root/project/dist-newstyle/build/x86_64-linux/ghc-8.6.2/hw-json-0.9.0.1/x/hw-json/opt/build/hw-json/hw-json-tmp/App/Commands/Types.o )
[2 of 7] Compiling App.Lens ( app/App/Lens.hs, /root/project/dist-newstyle/build/x86_64-linux/ghc-8.6.2/hw-json-0.9.0.1/x/hw-json/opt/build/hw-json/hw-json-tmp/App/Lens.o )
[3 of 7] Compiling App.Commands.Demo ( app/App/Commands/Demo.hs, /root/project/dist-newstyle/build/x86_64-linux/ghc-8.6.2/hw-json-0.9.0.1/x/hw-json/opt/build/hw-json/hw-json-tmp/App/Commands/Demo.o )
[2 of 2] Compiling Paths_hw_json ( /root/project/dist-newstyle/build/x86_64-linux/ghc-8.6.2/hw-json-0.9.0.1/b/bench/opt/build/bench/autogen/Paths_hw_json.hs, /root/project/dist-newstyle/build/x86_64-linux/ghc-8.6.2/hw-json-0.9.0.1/b/bench/opt/build/bench/bench-tmp/Paths_hw_json.o )
Linking /root/project/dist-newstyle/build/x86_64-linux/ghc-8.6.2/hw-json-0.9.0.1/b/bench/opt/build/bench/bench ...
[4 of 7] Compiling App.Commands.CreateIndex ( app/App/Commands/CreateIndex.hs, /root/project/dist-newstyle/build/x86_64-linux/ghc-8.6.2/hw-json-0.9.0.1/x/hw-json/opt/build/hw-json/hw-json-tmp/App/Commands/CreateIndex.o )
[5 of 7] Compiling App.Commands ( app/App/Commands.hs, /root/project/dist-newstyle/build/x86_64-linux/ghc-8.6.2/hw-json-0.9.0.1/x/hw-json/opt/build/hw-json/hw-json-tmp/App/Commands.o )
[ 2 of 11] Compiling HaskellWorks.Data.Json.Backend.Standard.Succinct.CursorSpec ( test/HaskellWorks/Data/Json/Backend/Standard/Succinct/CursorSpec.hs, /root/project/dist-newstyle/build/x86_64-linux/ghc-8.6.2/hw-json-0.9.0.1/t/hw-json-test/opt/build/hw-json-test/hw-json-test-tmp/HaskellWorks/Data/Json/Backend/Standard/Succinct/CursorSpec.o )
[6 of 7] Compiling Main ( app/Main.hs, /root/project/dist-newstyle/build/x86_64-linux/ghc-8.6.2/hw-json-0.9.0.1/x/hw-json/opt/build/hw-json/hw-json-tmp/Main.o )
[7 of 7] Compiling Paths_hw_json ( /root/project/dist-newstyle/build/x86_64-linux/ghc-8.6.2/hw-json-0.9.0.1/x/hw-json/opt/build/hw-json/autogen/Paths_hw_json.hs, /root/project/dist-newstyle/build/x86_64-linux/ghc-8.6.2/hw-json-0.9.0.1/x/hw-json/opt/build/hw-json/hw-json-tmp/Paths_hw_json.o )
Linking /root/project/dist-newstyle/build/x86_64-linux/ghc-8.6.2/hw-json-0.9.0.1/x/hw-json/opt/build/hw-json/hw-json ...
cabal: Failed to build test:hw-json-test from hw-json-0.9.0.1. The build
process was killed (i.e. SIGKILL). The typical reason for this is that there
is not enough memory available (e.g. the OS killed a process using lots of
memory).
Exited with code 1
```
<details><summary>Trac metadata</summary>
| Trac field | Value |
| ---------------------- | ------------ |
| Version | 8.6.2 |
| Type | Bug |
| TypeOfFailure | OtherFailure |
| Priority | normal |
| Resolution | Unresolved |
| Component | Compiler |
| Test case | |
| Differential revisions | |
| BlockedBy | |
| Related | |
| Blocking | |
| CC | |
| Operating system | |
| Architecture | |
</details>
<!-- {"blocked_by":[],"summary":"ghc-8.6.1 and ghc-8.6.2 use a log of memory","status":"New","operating_system":"","component":"Compiler","related":[],"milestone":"8.6.3","resolution":"Unresolved","owner":{"tag":"Unowned"},"version":"8.6.2","keywords":[],"differentials":[],"test_case":"","architecture":"","cc":[""],"type":"Bug","description":"Currently GHC uses a lot of memory to build a relatively small module and causes my CI to fail due to there being a 4G memory limit.\r\n\r\nThe source code can be found here:\r\n\r\nhttps://github.com/haskell-works/hw-json/tree/73368cee21dc72eedd5291ba689f9abf10e7fcd2\r\n\r\nThe problem module is here:\r\n\r\nhttps://github.com/haskell-works/hw-json/blob/73368cee21dc72eedd5291ba689f9abf10e7fcd2/test/HaskellWorks/Data/Json/Backend/Standard/Succinct/CursorSpec.hs\r\n\r\nThe build output follows:\r\n\r\n{{{\r\ncabal new-build --enable-tests --enable-benchmarks --project-file=\"cabal.project\" -j${CABAL_THREADS:-4} all\r\nBuild profile: -w ghc-8.6.2 -O2\r\nIn order, the following will be built (use -v for more details):\r\n - hw-json-0.9.0.1 (lib) (first run)\r\n - hw-json-0.9.0.1 (test:hw-json-test) (first run)\r\n - hw-json-0.9.0.1 (exe:hw-json) (first run)\r\n - hw-json-0.9.0.1 (bench:bench) (first run)\r\nConfiguring library for hw-json-0.9.0.1..\r\nPreprocessing library for hw-json-0.9.0.1..\r\nBuilding library for hw-json-0.9.0.1..\r\n[ 1 of 32] Compiling HaskellWorks.Data.Json.DecodeError ( src/HaskellWorks/Data/Json/DecodeError.hs, /root/project/dist-newstyle/build/x86_64-linux/ghc-8.6.2/hw-json-0.9.0.1/opt/build/HaskellWorks/Data/Json/DecodeError.o )\r\n[ 2 of 32] Compiling HaskellWorks.Data.Json.Internal.Backend.Standard.MakeIndex ( src/HaskellWorks/Data/Json/Internal/Backend/Standard/MakeIndex.hs, /root/project/dist-newstyle/build/x86_64-linux/ghc-8.6.2/hw-json-0.9.0.1/opt/build/HaskellWorks/Data/Json/Internal/Backend/Standard/MakeIndex.o )\r\n[ 3 of 32] Compiling HaskellWorks.Data.Json.Internal.Index ( src/HaskellWorks/Data/Json/Internal/Index.hs, /root/project/dist-newstyle/build/x86_64-linux/ghc-8.6.2/hw-json-0.9.0.1/opt/build/HaskellWorks/Data/Json/Internal/Index.o )\r\n[ 4 of 32] Compiling HaskellWorks.Data.Json.Internal.PartialIndex ( src/HaskellWorks/Data/Json/Internal/PartialIndex.hs, /root/project/dist-newstyle/build/x86_64-linux/ghc-8.6.2/hw-json-0.9.0.1/opt/build/HaskellWorks/Data/Json/Internal/PartialIndex.o )\r\n[ 5 of 32] Compiling HaskellWorks.Data.Json.Internal.Token.Types ( src/HaskellWorks/Data/Json/Internal/Token/Types.hs, /root/project/dist-newstyle/build/x86_64-linux/ghc-8.6.2/hw-json-0.9.0.1/opt/build/HaskellWorks/Data/Json/Internal/Token/Types.o )\r\n[ 6 of 32] Compiling HaskellWorks.Data.Json.Internal.Backend.Standard.Token.Tokenize ( src/HaskellWorks/Data/Json/Internal/Backend/Standard/Token/Tokenize.hs, /root/project/dist-newstyle/build/x86_64-linux/ghc-8.6.2/hw-json-0.9.0.1/opt/build/HaskellWorks/Data/Json/Internal/Backend/Standard/Token/Tokenize.o )\r\n[ 7 of 32] Compiling HaskellWorks.Data.Json.Internal.Token ( src/HaskellWorks/Data/Json/Internal/Token.hs, /root/project/dist-newstyle/build/x86_64-linux/ghc-8.6.2/hw-json-0.9.0.1/opt/build/HaskellWorks/Data/Json/Internal/Token.o )\r\n[ 8 of 32] Compiling HaskellWorks.Data.Json.Internal.Value ( src/HaskellWorks/Data/Json/Internal/Value.hs, /root/project/dist-newstyle/build/x86_64-linux/ghc-8.6.2/hw-json-0.9.0.1/opt/build/HaskellWorks/Data/Json/Internal/Value.o )\r\n[ 9 of 32] Compiling HaskellWorks.Data.Json.Internal.Word8 ( src/HaskellWorks/Data/Json/Internal/Word8.hs, /root/project/dist-newstyle/build/x86_64-linux/ghc-8.6.2/hw-json-0.9.0.1/opt/build/HaskellWorks/Data/Json/Internal/Word8.o )\r\n[10 of 32] Compiling HaskellWorks.Data.Json.Internal.Word64 ( src/HaskellWorks/Data/Json/Internal/Word64.hs, /root/project/dist-newstyle/build/x86_64-linux/ghc-8.6.2/hw-json-0.9.0.1/opt/build/HaskellWorks/Data/Json/Internal/Word64.o )\r\n[11 of 32] Compiling HaskellWorks.Data.Json.Internal.CharLike ( src/HaskellWorks/Data/Json/Internal/CharLike.hs, /root/project/dist-newstyle/build/x86_64-linux/ghc-8.6.2/hw-json-0.9.0.1/opt/build/HaskellWorks/Data/Json/Internal/CharLike.o )\r\n[12 of 32] Compiling HaskellWorks.Data.Json.Internal.Backend.Standard.Blank ( src/HaskellWorks/Data/Json/Internal/Backend/Standard/Blank.hs, /root/project/dist-newstyle/build/x86_64-linux/ghc-8.6.2/hw-json-0.9.0.1/opt/build/HaskellWorks/Data/Json/Internal/Backend/Standard/Blank.o )\r\n[13 of 32] Compiling HaskellWorks.Data.Json.Internal.Backend.Standard.BlankedJson ( src/HaskellWorks/Data/Json/Internal/Backend/Standard/BlankedJson.hs, /root/project/dist-newstyle/build/x86_64-linux/ghc-8.6.2/hw-json-0.9.0.1/opt/build/HaskellWorks/Data/Json/Internal/Backend/Standard/BlankedJson.o )\r\n[14 of 32] Compiling HaskellWorks.Data.Json.Internal.Backend.Standard.ToInterestBits64 ( src/HaskellWorks/Data/Json/Internal/Backend/Standard/ToInterestBits64.hs, /root/project/dist-newstyle/build/x86_64-linux/ghc-8.6.2/hw-json-0.9.0.1/opt/build/HaskellWorks/Data/Json/Internal/Backend/Standard/ToInterestBits64.o )\r\n[15 of 32] Compiling HaskellWorks.Data.Json.Internal.Backend.Standard.ToBalancedParens64 ( src/HaskellWorks/Data/Json/Internal/Backend/Standard/ToBalancedParens64.hs, /root/project/dist-newstyle/build/x86_64-linux/ghc-8.6.2/hw-json-0.9.0.1/opt/build/HaskellWorks/Data/Json/Internal/Backend/Standard/ToBalancedParens64.o )\r\n[16 of 32] Compiling HaskellWorks.Data.Json.Internal.Backend.Standard.IbBp ( src/HaskellWorks/Data/Json/Internal/Backend/Standard/IbBp.hs, /root/project/dist-newstyle/build/x86_64-linux/ghc-8.6.2/hw-json-0.9.0.1/opt/build/HaskellWorks/Data/Json/Internal/Backend/Standard/IbBp.o )\r\n[17 of 32] Compiling HaskellWorks.Data.Json.Backend.Standard.SemiIndex ( src/HaskellWorks/Data/Json/Backend/Standard/SemiIndex.hs, /root/project/dist-newstyle/build/x86_64-linux/ghc-8.6.2/hw-json-0.9.0.1/opt/build/HaskellWorks/Data/Json/Backend/Standard/SemiIndex.o )\r\n[18 of 32] Compiling HaskellWorks.Data.Json.Backend.Simple.SemiIndex ( src/HaskellWorks/Data/Json/Backend/Simple/SemiIndex.hs, /root/project/dist-newstyle/build/x86_64-linux/ghc-8.6.2/hw-json-0.9.0.1/opt/build/HaskellWorks/Data/Json/Backend/Simple/SemiIndex.o )\r\n[19 of 32] Compiling HaskellWorks.Data.Json.LightJson ( src/HaskellWorks/Data/Json/LightJson.hs, /root/project/dist-newstyle/build/x86_64-linux/ghc-8.6.2/hw-json-0.9.0.1/opt/build/HaskellWorks/Data/Json/LightJson.o )\r\n[20 of 32] Compiling HaskellWorks.Data.Json.PartialValue ( src/HaskellWorks/Data/Json/PartialValue.hs, /root/project/dist-newstyle/build/x86_64-linux/ghc-8.6.2/hw-json-0.9.0.1/opt/build/HaskellWorks/Data/Json/PartialValue.o )\r\n[21 of 32] Compiling HaskellWorks.Data.Json.Type ( src/HaskellWorks/Data/Json/Type.hs, /root/project/dist-newstyle/build/x86_64-linux/ghc-8.6.2/hw-json-0.9.0.1/opt/build/HaskellWorks/Data/Json/Type.o )\r\n[22 of 32] Compiling HaskellWorks.Data.Json.Backend.Standard.Cursor ( src/HaskellWorks/Data/Json/Backend/Standard/Cursor.hs, /root/project/dist-newstyle/build/x86_64-linux/ghc-8.6.2/hw-json-0.9.0.1/opt/build/HaskellWorks/Data/Json/Backend/Standard/Cursor.o )\r\n[23 of 32] Compiling HaskellWorks.Data.Json.Cursor ( src/HaskellWorks/Data/Json/Cursor.hs, /root/project/dist-newstyle/build/x86_64-linux/ghc-8.6.2/hw-json-0.9.0.1/opt/build/HaskellWorks/Data/Json/Cursor.o )\r\n[24 of 32] Compiling HaskellWorks.Data.Json.Internal.Backend.Standard.Cursor.Token ( src/HaskellWorks/Data/Json/Internal/Backend/Standard/Cursor/Token.hs, /root/project/dist-newstyle/build/x86_64-linux/ghc-8.6.2/hw-json-0.9.0.1/opt/build/HaskellWorks/Data/Json/Internal/Backend/Standard/Cursor/Token.o )\r\n[25 of 32] Compiling HaskellWorks.Data.Json.Value ( src/HaskellWorks/Data/Json/Value.hs, /root/project/dist-newstyle/build/x86_64-linux/ghc-8.6.2/hw-json-0.9.0.1/opt/build/HaskellWorks/Data/Json/Value.o )\r\n[26 of 32] Compiling HaskellWorks.Data.Json.FromValue ( src/HaskellWorks/Data/Json/FromValue.hs, /root/project/dist-newstyle/build/x86_64-linux/ghc-8.6.2/hw-json-0.9.0.1/opt/build/HaskellWorks/Data/Json/FromValue.o )\r\n[27 of 32] Compiling HaskellWorks.Data.Json.Backend.Standard.LoadCursor ( src/HaskellWorks/Data/Json/Backend/Standard/LoadCursor.hs, /root/project/dist-newstyle/build/x86_64-linux/ghc-8.6.2/hw-json-0.9.0.1/opt/build/HaskellWorks/Data/Json/Backend/Standard/LoadCursor.o )\r\n[28 of 32] Compiling HaskellWorks.Data.Json.LoadCursor ( src/HaskellWorks/Data/Json/LoadCursor.hs, /root/project/dist-newstyle/build/x86_64-linux/ghc-8.6.2/hw-json-0.9.0.1/opt/build/HaskellWorks/Data/Json/LoadCursor.o )\r\n[29 of 32] Compiling HaskellWorks.Data.Json.Backend.Standard.Load ( src/HaskellWorks/Data/Json/Backend/Standard/Load.hs, /root/project/dist-newstyle/build/x86_64-linux/ghc-8.6.2/hw-json-0.9.0.1/opt/build/HaskellWorks/Data/Json/Backend/Standard/Load.o )\r\n[30 of 32] Compiling HaskellWorks.Data.Json.Load ( src/HaskellWorks/Data/Json/Load.hs, /root/project/dist-newstyle/build/x86_64-linux/ghc-8.6.2/hw-json-0.9.0.1/opt/build/HaskellWorks/Data/Json/Load.o )\r\n[31 of 32] Compiling HaskellWorks.Data.Json ( src/HaskellWorks/Data/Json.hs, /root/project/dist-newstyle/build/x86_64-linux/ghc-8.6.2/hw-json-0.9.0.1/opt/build/HaskellWorks/Data/Json.o )\r\n[32 of 32] Compiling Paths_hw_json ( /root/project/dist-newstyle/build/x86_64-linux/ghc-8.6.2/hw-json-0.9.0.1/opt/build/autogen/Paths_hw_json.hs, /root/project/dist-newstyle/build/x86_64-linux/ghc-8.6.2/hw-json-0.9.0.1/opt/build/Paths_hw_json.o )\r\nConfiguring test suite 'hw-json-test' for hw-json-0.9.0.1..\r\nConfiguring benchmark 'bench' for hw-json-0.9.0.1..\r\nConfiguring executable 'hw-json' for hw-json-0.9.0.1..\r\nPreprocessing test suite 'hw-json-test' for hw-json-0.9.0.1..\r\nBuilding test suite 'hw-json-test' for hw-json-0.9.0.1..\r\nPreprocessing executable 'hw-json' for hw-json-0.9.0.1..\r\nPreprocessing benchmark 'bench' for hw-json-0.9.0.1..\r\nBuilding benchmark 'bench' for hw-json-0.9.0.1..\r\nBuilding executable 'hw-json' for hw-json-0.9.0.1..\r\n[ 1 of 11] Compiling HaskellWorks.Data.Json.Backend.Standard.Succinct.Cursor.InterestBitsSpec ( test/HaskellWorks/Data/Json/Backend/Standard/Succinct/Cursor/InterestBitsSpec.hs, /root/project/dist-newstyle/build/x86_64-linux/ghc-8.6.2/hw-json-0.9.0.1/t/hw-json-test/opt/build/hw-json-test/hw-json-test-tmp/HaskellWorks/Data/Json/Backend/Standard/Succinct/Cursor/InterestBitsSpec.o )\r\n[1 of 2] Compiling Main ( bench/Main.hs, /root/project/dist-newstyle/build/x86_64-linux/ghc-8.6.2/hw-json-0.9.0.1/b/bench/opt/build/bench/bench-tmp/Main.o )\r\n[1 of 7] Compiling App.Commands.Types ( app/App/Commands/Types.hs, /root/project/dist-newstyle/build/x86_64-linux/ghc-8.6.2/hw-json-0.9.0.1/x/hw-json/opt/build/hw-json/hw-json-tmp/App/Commands/Types.o )\r\n[2 of 7] Compiling App.Lens ( app/App/Lens.hs, /root/project/dist-newstyle/build/x86_64-linux/ghc-8.6.2/hw-json-0.9.0.1/x/hw-json/opt/build/hw-json/hw-json-tmp/App/Lens.o )\r\n[3 of 7] Compiling App.Commands.Demo ( app/App/Commands/Demo.hs, /root/project/dist-newstyle/build/x86_64-linux/ghc-8.6.2/hw-json-0.9.0.1/x/hw-json/opt/build/hw-json/hw-json-tmp/App/Commands/Demo.o )\r\n[2 of 2] Compiling Paths_hw_json ( /root/project/dist-newstyle/build/x86_64-linux/ghc-8.6.2/hw-json-0.9.0.1/b/bench/opt/build/bench/autogen/Paths_hw_json.hs, /root/project/dist-newstyle/build/x86_64-linux/ghc-8.6.2/hw-json-0.9.0.1/b/bench/opt/build/bench/bench-tmp/Paths_hw_json.o )\r\nLinking /root/project/dist-newstyle/build/x86_64-linux/ghc-8.6.2/hw-json-0.9.0.1/b/bench/opt/build/bench/bench ...\r\n[4 of 7] Compiling App.Commands.CreateIndex ( app/App/Commands/CreateIndex.hs, /root/project/dist-newstyle/build/x86_64-linux/ghc-8.6.2/hw-json-0.9.0.1/x/hw-json/opt/build/hw-json/hw-json-tmp/App/Commands/CreateIndex.o )\r\n[5 of 7] Compiling App.Commands ( app/App/Commands.hs, /root/project/dist-newstyle/build/x86_64-linux/ghc-8.6.2/hw-json-0.9.0.1/x/hw-json/opt/build/hw-json/hw-json-tmp/App/Commands.o )\r\n[ 2 of 11] Compiling HaskellWorks.Data.Json.Backend.Standard.Succinct.CursorSpec ( test/HaskellWorks/Data/Json/Backend/Standard/Succinct/CursorSpec.hs, /root/project/dist-newstyle/build/x86_64-linux/ghc-8.6.2/hw-json-0.9.0.1/t/hw-json-test/opt/build/hw-json-test/hw-json-test-tmp/HaskellWorks/Data/Json/Backend/Standard/Succinct/CursorSpec.o )\r\n[6 of 7] Compiling Main ( app/Main.hs, /root/project/dist-newstyle/build/x86_64-linux/ghc-8.6.2/hw-json-0.9.0.1/x/hw-json/opt/build/hw-json/hw-json-tmp/Main.o )\r\n[7 of 7] Compiling Paths_hw_json ( /root/project/dist-newstyle/build/x86_64-linux/ghc-8.6.2/hw-json-0.9.0.1/x/hw-json/opt/build/hw-json/autogen/Paths_hw_json.hs, /root/project/dist-newstyle/build/x86_64-linux/ghc-8.6.2/hw-json-0.9.0.1/x/hw-json/opt/build/hw-json/hw-json-tmp/Paths_hw_json.o )\r\nLinking /root/project/dist-newstyle/build/x86_64-linux/ghc-8.6.2/hw-json-0.9.0.1/x/hw-json/opt/build/hw-json/hw-json ...\r\ncabal: Failed to build test:hw-json-test from hw-json-0.9.0.1. The build\r\nprocess was killed (i.e. SIGKILL). The typical reason for this is that there\r\nis not enough memory available (e.g. the OS killed a process using lots of\r\nmemory).\r\n\r\nExited with code 1\r\n}}}\r\n\r\n","type_of_failure":"OtherFailure","blocking":[]} -->8.6.3Alp MestanogullariAlp Mestanogullarihttps://gitlab.haskell.org/ghc/ghc/-/issues/16884SpecConstr generating bad specialisations2022-04-28T19:10:16ZSimon Peyton JonesSpecConstr generating bad specialisationsTry SpecConstr on this code (`determ006`):
```
module Lib where
foo :: Int -> Int
foo n =
go (Just n) (Just (6::Int))
where
go Nothing (Just x) = go (Just 10) (Just x)
go (Just n) (Just x)
| n <= 0 = 0
| otherwise = g...Try SpecConstr on this code (`determ006`):
```
module Lib where
foo :: Int -> Int
foo n =
go (Just n) (Just (6::Int))
where
go Nothing (Just x) = go (Just 10) (Just x)
go (Just n) (Just x)
| n <= 0 = 0
| otherwise = go (Just (n-1)) (Just x)
```
There are really just two call patterns to `go`:
```
(A) forall (n::Int) (m::Int#). go (Just n) (Just (I# m))
(B) forall (n::Int#) (m::Int#). go (Just (I# n)) (Just (I# m))
```
If `go` could be called from somewhere else (which in fact it can't)
then the recursive call in the RHS would also give this call pattern
```
(C) forall (n::Int#) (m::Int). go (Just (I# n)) (Just m)
```
In fact GHC makes these three specialisations:
```
RULES: "SC:$wgo1" [2] -- A bit like (B)
forall (sc_s2wc :: GHC.Prim.Int#).
$wgo_s2vk (GHC.Maybe.Just @ Int (GHC.Types.I# sc_s2wc))
(GHC.Maybe.Just @ Int lvl_s2uv)
= $s$wgo_s2wh sc_s2wc
"SC:$wgo2" [2] -- This is (C)
forall (sc_s2we :: Int) (sc_s2wd :: GHC.Prim.Int#).
$wgo_s2vk (GHC.Maybe.Just @ Int (GHC.Types.I# sc_s2wd))
(GHC.Maybe.Just @ Int sc_s2we)
= $s$wgo_s2wi sc_s2we sc_s2wd
"SC:$wgo0" [2] -- A bit like (A)
forall (sc_s2w8 :: Int).
$wgo_s2vk (GHC.Maybe.Just @ Int sc_s2w8)
(GHC.Maybe.Just @ Int lvl_s2uv)
= $s$wgo_s2wb sc_s2w8]
```
Notice that the first and last of these **mentions the un-forall'd `lvl_s2uv` on
the LHS**. Even worse, `lvl_s2uv` isn't in scope at that point (#16192).
Even if `lvl_s2uv` is in scope, I'm afraid that during matching we'll
only match when the corresponding argument is precisely equal to `lvl_s2uv` which
is terribly over-specialised.
Incidentally, we don't for a _particular_ `Int#` literal like
```
go (Just (I# 6#)) ...
```
and I think that's right: there are too many such literals.
Here's the code immediately pre-SpecConstr
```
Rec {
$wgo_s2vk :: Maybe Int -> Maybe Int -> GHC.Prim.Int#
$wgo_s2vk
= \ (w_s2vf :: Maybe Int) (w_s2vg :: Maybe Int) ->
case w_s2vf of {
Nothing ->
case w_s2vg of wild_X8 [Dmd=<L,A>] {
Nothing -> case lvl_s2vI of wild_00 { };
Just x_aY0 -> $wgo_s2vk lvl_s2us wild_X8
};
Just n_aY1 [Dmd=<S(S),U(U)>] ->
case w_s2vg of wild_X9 [Dmd=<L,A>] {
Nothing -> case lvl_s2vI of wild_00 { };
Just x_aY2 ->
case n_aY1 of { GHC.Types.I# x_a2tS [Dmd=<S,U>] ->
case GHC.Prim.<=# x_a2tS 0# of {
__DEFAULT ->
$wgo_s2vk
(GHC.Maybe.Just @ Int (GHC.Types.I# (GHC.Prim.-# x_a2tS 1#)))
wild_X9;
1# -> 0#
}
}
}
}
end Rec }
lvl_s2uv :: Int
lvl_s2uv = GHC.Types.I# 6#
lvl_s2uw :: Maybe Int
lvl_s2uw = GHC.Maybe.Just @ Int lvl_s2uv
foo :: Int -> Int
foo = \ (n_aX7 :: Int) ->
case $wgo_s2vk (GHC.Maybe.Just @ Int n_aX7) lvl_s2uw of ww_s2vj
{ __DEFAULT -> GHC.Types.I# ww_s2vj }
```
And post-SpecConstr -- note the uses of `lvl_s2uv`:
```
Rec {
$wgo_s2vk :: Maybe Int -> Maybe Int -> GHC.Prim.Int#
[RULES: "SC:$wgo1" [2]
forall (sc_s2wc :: GHC.Prim.Int#).
$wgo_s2vk (GHC.Maybe.Just @ Int (GHC.Types.I# sc_s2wc))
(GHC.Maybe.Just @ Int lvl_s2uv)
= $s$wgo_s2wh sc_s2wc
"SC:$wgo2" [2]
forall (sc_s2we :: Int) (sc_s2wd :: GHC.Prim.Int#).
$wgo_s2vk (GHC.Maybe.Just @ Int (GHC.Types.I# sc_s2wd))
(GHC.Maybe.Just @ Int sc_s2we)
= $s$wgo_s2wi sc_s2we sc_s2wd
"SC:$wgo0" [2]
forall (sc_s2w8 :: Int).
$wgo_s2vk (GHC.Maybe.Just @ Int sc_s2w8)
(GHC.Maybe.Just @ Int lvl_s2uv)
= $s$wgo_s2wb sc_s2w8]
$wgo_s2vk
= \ (w_s2vf :: Maybe Int) (w_s2vg :: Maybe Int) ->
case w_s2vf of {
Nothing ->
case w_s2vg of wild_X8 [Dmd=<L,A>] {
Nothing -> case lvl_s2vI of wild_00 { };
Just x_aY0 -> $wgo_s2vk lvl_s2us wild_X8
};
Just n_aY1 [Dmd=<S(S),U(U)>] ->
case w_s2vg of wild_X9 [Dmd=<L,A>] {
Nothing -> case lvl_s2vI of wild_00 { };
Just x_aY2 ->
case n_aY1 of { GHC.Types.I# x_a2tS [Dmd=<S,U>] ->
case GHC.Prim.<=# x_a2tS 0# of {
__DEFAULT ->
$wgo_s2vk
(GHC.Maybe.Just @ Int (GHC.Types.I# (GHC.Prim.-# x_a2tS 1#)))
wild_X9;
1# -> 0#
}
}
}
}
end Rec }
lvl_s2uv :: Int
lvl_s2uv = GHC.Types.I# 6#
lvl_s2uw :: Maybe Int
lvl_s2uw = GHC.Maybe.Just @ Int lvl_s2uv
foo :: Int -> Int
foo
= \ (n_aX7 :: Int) ->
case $wgo_s2vk (GHC.Maybe.Just @ Int n_aX7) lvl_s2uw of ww_s2vj
{ __DEFAULT ->
GHC.Types.I# ww_s2vj
}
```https://gitlab.haskell.org/ghc/ghc/-/issues/17592SpecConstr rule base woes2020-01-15T20:36:10ZSebastian GrafSpecConstr rule base woesConsider the following stream fusion approach harnessing the type class specialiser (and also constructor-pattern specialisation for subsequently specialising `loop`):
```hs
{-# LANGUAGE TypeFamilies, FlexibleContexts, BangPatterns #-}
...Consider the following stream fusion approach harnessing the type class specialiser (and also constructor-pattern specialisation for subsequently specialising `loop`):
```hs
{-# LANGUAGE TypeFamilies, FlexibleContexts, BangPatterns #-}
data Step iter
= Yield !iter !(Item iter)
| Skip !iter
| Done
class Iterator iter where
type Item iter
next :: iter -> Step iter
data Singleton a = Empty' | Singleton !a
singletonS :: a -> Singleton a
singletonS = Singleton
instance Iterator (Singleton a) where
type Item (Singleton a) = a
next Empty' = Done
next (Singleton a) = Yield Empty' a
data EnumFromTo a = EnumFromTo !a !a
enumFromToS :: a -> a -> EnumFromTo a
enumFromToS = EnumFromTo
instance (Ord a, Num a) => Iterator (EnumFromTo a) where
type Item (EnumFromTo a) = a
next (EnumFromTo i high)
| i > high = Done
| otherwise = Yield (EnumFromTo (i + 1) high) i
sumS :: (Num (Item iter), Iterator iter) => iter -> Item iter
sumS = loop 0
where
loop !total iter1 =
case next iter1 of
Done -> total
Skip iter2 -> loop total iter2
Yield iter2 x -> loop (total + x) iter2
-- {-# NOINLINE[99] loop #-} -- comment in for 10x speedup
{-# INLINE sumS #-}
data ConcatMap i o = ConcatMap !(Maybe o) !(Item i -> o) !i
concatMapS :: (Item i -> o) -> i -> ConcatMap i o
concatMapS = ConcatMap Nothing
instance (Iterator i, Iterator o) => Iterator (ConcatMap i o) where
type Item (ConcatMap i o) = Item o
next (ConcatMap Nothing f i) = case next i of
Skip i' -> Skip (ConcatMap Nothing f i')
Yield i' x -> Skip (ConcatMap (Just (f x)) f i')
Done -> Done
next (ConcatMap (Just o) f i) = case next o of
Skip o' -> Skip (ConcatMap (Just o') f i)
Yield o' x -> Yield (ConcatMap (Just o') f i) x
Done -> Skip (ConcatMap Nothing f i)
ex1 :: Int -> Int
ex1
= sumS
. concatMapS (\a -> singletonS (a*a))
. enumFromToS 1
{-# NOINLINE ex1 #-}
main = print (ex1 99999999)
```
If the `{-# NOINLINE[99] loop #-}` pragma is commented in, `ex1` will compile down to this perfect bit of code:
```
Main.main_$s$wloop
= \ (sc_s4GJ :: GHC.Prim.Int#)
(sc1_s4GI :: GHC.Prim.Int#)
(sc2_s4GH :: GHC.Prim.Int#) ->
case GHC.Prim.># sc1_s4GI sc_s4GJ of {
__DEFAULT ->
Main.main_$s$wloop
sc_s4GJ
(GHC.Prim.+# sc1_s4GI 1#)
(GHC.Prim.+# sc2_s4GH (GHC.Prim.*# sc1_s4GI sc1_s4GI));
1# -> sc2_s4GH
}
```
Beautiful!
Without the `NOINLINE` pragma, not so much, though. Here's the Core:
```
Main.main_$s$wloop [Occ=LoopBreaker]
:: GHC.Prim.Int#
-> GHC.Prim.Int#
-> (Item (EnumFromTo Int) -> Singleton Int)
-> Singleton Int
-> GHC.Prim.Int#
-> GHC.Prim.Int#
[GblId,
Arity=5,
Caf=NoCafRefs,
Str=<S,U><S,U><L,C(U)><S,1*U><S,U>,
Unf=OtherCon []]
Main.main_$s$wloop
= \ (sc_s4FQ :: GHC.Prim.Int#)
(sc1_s4FP :: GHC.Prim.Int#)
(sc2_s4FO :: Item (EnumFromTo Int) -> Singleton Int)
(sc3_s4FN :: Singleton Int)
(sc4_s4FM :: GHC.Prim.Int#) ->
case sc3_s4FN of {
Empty' ->
case GHC.Prim.># sc1_s4FP sc_s4FQ of {
__DEFAULT ->
Main.main_$s$wloop
sc_s4FQ
(GHC.Prim.+# sc1_s4FP 1#)
sc2_s4FO
(sc2_s4FO
((GHC.Types.I# sc1_s4FP)
`cast` (Sub (Sym (Main.D:R:ItemEnumFromTo[0] <Int>_N))
:: Int ~R# Item (EnumFromTo Int))))
sc4_s4FM;
1# -> sc4_s4FM
};
Singleton a_a1z4 ->
case a_a1z4 of { GHC.Types.I# y_s4Dg ->
Main.main_$s$wloop
sc_s4FQ
sc1_s4FP
sc2_s4FO
(Main.Empty' @ Int)
(GHC.Prim.+# sc4_s4FM y_s4Dg)
}
}
```
Long story short, constructor specialisation failed to specialise `$wloop` for some higher-order arguments (the function of type `Item (EnumFromTo Int) -> Singleton Int` in particular). That results in a slow-down of a factor of 10 and it goes from constant space to allocating huge amounts of memory (still constant residency, though).
# Why?
**Edit: Don't bother reading this section yet, it's incoherent and inaccurate. Suffice it to say, the conclusion is accurate: Even though we specialise for the stronger call pattern, we don't rewrite to it because the rewrite rule is not attached to the strictly weaker specialisation.**
Looking at the difference in `-dverbose-core2core`, the first real divergence is in the output from desugaring after optimisation, where the inner `loop` is specialised for the type arguments and dictionaries of the outer `sumS` function in the version without `NOINLINE`.
Then after the initial simplifier phase we get the first digression because `loop` was specialised for the particular types it was called at, whereas it wasn't in the `NOINLINE` version (which makes sense).
Now, the critical moment is when `loop` becomes too large to inline into `ex1` in the version *without* `NOINLINE`. But `ex1` calls `loop` with quite interesting arguments, those we want to specialise for (the particular `ConcatMap` datum in particular)! In code:
```
f :: Int -> Singleton Int -- same function for both
f ...
inner :: Int -> EnumFromToInt Int -- the input stream from which ConcatMap draws elements
inner ...
-- without NOINLINE
loop (ConcatMap x f inner) = HUGE and recursive
ex1 n = loop (ConcatMap Nothing f (inner n))
-- with NOINLINE[99], loop gets inlined after the initial phase
ex1 n = joinrec loop' (ConcatMap x f (inner n)) = <HUGE and recursive> in loop' (ConcatMap Nothing f (inner n))
```
Then WW kicks in. `loop` (and `loop'`) is deeply strict in the `ConcatMap` we pass it, but it refuses to 'unpack' (i.e. specialise) for `ConcatMap`'s `f`. Note also that it will WW `loop`, `ex1` and `loop'`:
```
-- without NOINLINE
$wloop x f <unpacked components from inner> = HUGE and recursive
loop (ConcatMap x f (.. unpack components from inner ...)) = $wloop x f <unpacked components from inner>
ex1 n = $wloop (ConcatMap Nothing f (inner n))
-- with NOINLINE
$wex1 n = joinrec $wloop' (ConcatMap x f inner) = <HUGE and recursive> in $wloop' (ConcatMap Nothing f (inner n))
ex1 (I# n) = $wex1 n
```
Note how in the NOINLINE case this separated the definition of `f` from its usage by a lambda binder, so the subsequent SpecConstr pass is unable to figure out what to specialise for. It can't look through `$fw`. I think this is the same or at least a similar effect that @nomeata experienced in #14068 and accounted for in #14844.
Or at least that's my working hypothesis, I'll edit this post accordingly tomorrow when I had time to investigate.https://gitlab.haskell.org/ghc/ghc/-/issues/18837Improve interaction of exitification with spec constr2020-11-09T14:01:11ZharendraImprove interaction of exitification with spec constr## Summary
Exitification pass happens before spec constr pass. It moves exit path code out of a recursive function. However, this impacts the efficacy of the later spec constr pass.
See https://github.com/composewell/streamly/issues/70...## Summary
Exitification pass happens before spec constr pass. It moves exit path code out of a recursive function. However, this impacts the efficacy of the later spec constr pass.
See https://github.com/composewell/streamly/issues/703 for details.
## Steps to reproduce
1. `git clone https://github.com/composewell/streamly.git`
2. `git checkout ghc-spec-constr-keen`
3. `cabal build streamly`
4. `ghc -ddump-simpl -ddump-to-file -dsuppress-all -O2 -fmax-worker-args=16 -fspec-constr-recursive=16 -funfolding-use-threshold=600 fold-bench.hs`
5. time ./fold-bench +RTS -s
6. Examine fold-bench.dump-simpl look for `W8#`.
We can see a W8# being passed to the join point $wstep3_s5F2:
```
jump $wstep3_s5F2
sc9_s5W3
sc8_s5W4
(PlainPtr sc7_s5W5)
sc6_s5W6
()
(W8# ipv8_a4F5)
sc_s5Wc
```
This join point then passes it to an exit point:
```
1# ->
jump exit1_X19
ww_s5EQ ww1_s5EV ww2_s5EW ww3_s5EX ww4_s5F0 w_s5EG w1_s5EH
```
The exit point examines this W8#:
```
exit1_X19 ww_s5EQ
ww1_s5EV
ww2_s5EW
ww3_s5EX
ww4_s5F0
w_s5EG
w1_s5EH
= case w_s5EG of { W8# x_a4tc ->
```
The W8# constructor is not removed by the spec-constr pass because the W8# constructor is not being examined by `$wstep3_s5F2`. The code examining the W8# has been moved to the exit point by the exitification pass which happens before the spec-constr pass. So it becomes opaque to the spec-constr pass and not removed.
We can use `-fspec-constr-keen` option in the compilation step above and see that the W8# is removed by it. However, using that option creates regressions in other benchmarks, it is not always good.
Another option is to do exitification pass after the spec-constr pass. See [this note](https://gitlab.haskell.org/ghc/ghc/-/blob/a9ce159ba58ca7e8946b46e19b1361588b677a26/compiler/GHC/Core/Opt/Exitify.hs#L451). We would not have this issue if we use the option D suggested in this note i.e. put exitification before the final simplifier pass. However, this is also not the best possible option because exitify before spec-constr + -fspec-constr-keen generates much better code and has 2x better performance.
We can checkout the branch `ghc-spec-constr-before-exitify` and repeat the steps 3-6 above to see that the W8# constructor is gone when exitify is done after spec-constr.
## Expected behavior
The point of this issue is to explore if we can keep exitify before spec-constr but do what -fspec-constr-keen does automatically without enabling it always but only surgically in cases like this. Can we make spec-constr keen through exit join points only?
The effect of removing the W8# in the above examples is 4x improvement because it in the fast path, called for every element of the stream. Depending on the use case we can get very significant gains if we can do spec-constr smartly to get the best results.
## Environment
* GHC version used:
8.10.2 branch
Optional:
* Operating System: macOS
* System Architecture: x86_64https://gitlab.haskell.org/ghc/ghc/-/issues/19695Exponential memory use when using lots of <*>2022-12-21T16:26:50ZmaxigitExponential memory use when using lots of <*>## Summary
Some code with the shape `F <$> a1 <*> a2 <*> a3 <*> ...` takes forever or lots of memory (up to 50G) or end up in stack overflow (depending on how long I wait) when using optimisation.
## Steps to reproduce
I've created a...## Summary
Some code with the shape `F <$> a1 <*> a2 <*> a3 <*> ...` takes forever or lots of memory (up to 50G) or end up in stack overflow (depending on how long I wait) when using optimisation.
## Steps to reproduce
I've created a single file project which reproduce the issue. It can be found on [github]( https://github.com/maxigit/ghc-bug)
Clone the project.
run `stack build` and wait ...
The issue seems to be related to the use of `CountryCode` (the `shCountry` field) which is a enum with around hundred constructor.
- Disabling the optimisation fixes the problem
- Removing a few field in the form fixes the problem too
## Expected behavior
The compilation should finish in a few seconds not crash after hours
## Environment
* GHC version used:
8.10.4 8.10.3 8.8.4 8.4.4
Optional:
* Operating System:
Linux
* System Architecture:https://gitlab.haskell.org/ghc/ghc/-/issues/19796SpecConstr passes unused parameters2021-09-08T13:55:13ZSimon Peyton JonesSpecConstr passes unused parametersConsider
```
f x y = case x of
a : as -> f as y
[] -> True
boo p ps = f (p:ps) 'v'
```
Notice that `y` is unused, and so is `a` in the pattern match.
Compile with -O2, and we get (as the output of SpecConstr):
`...Consider
```
f x y = case x of
a : as -> f as y
[] -> True
boo p ps = f (p:ps) 'v'
```
Notice that `y` is unused, and so is `a` in the pattern match.
Compile with -O2, and we get (as the output of SpecConstr):
```
Rec {
-- RHS size: {terms: 5, types: 7, coercions: 0, joins: 0/0}
Foo.f_$s$wf [Occ=LoopBreaker] :: forall a. a -> [a] -> Bool
[GblId, Arity=2, Caf=NoCafRefs, Str=<L,A><S,1*U>, Unf=OtherCon []]
Foo.f_$s$wf
= \ (@ a_aud) _ [Occ=Dead] (sc1_sv3 :: [a_aud]) ->
Foo.$wf @ a_aud @ Char sc1_sv3
-- RHS size: {terms: 10, types: 13, coercions: 0, joins: 0/0}
Foo.$wf [InlPrag=NOUSERINLINE[2], Occ=LoopBreaker]
:: forall a t. [a] -> Bool
[GblId, Arity=1, Caf=NoCafRefs, Str=<S,1*U>, Unf=OtherCon []]
Foo.$wf
= \ (@ a_suO) (@ t_suP) (w_suQ :: [a_suO]) ->
case w_suQ of {
[] -> GHC.Types.True;
: a1_agf as_agg -> Foo.$wf @ a_suO @ t_suP as_agg
}
end Rec }
-- RHS size: {terms: 6, types: 9, coercions: 0, joins: 0/0}
f [InlPrag=NOUSERINLINE[2]] :: forall a t. [a] -> t -> Bool
[GblId,
Arity=2,
Caf=NoCafRefs,
Str=<S,1*U><L,A>,
Unf=Unf{Src=InlineStable, TopLvl=True, Value=True, ConLike=True,
WorkFree=True, Expandable=True,
Guidance=ALWAYS_IF(arity=2,unsat_ok=True,boring_ok=True)
Tmpl= \ (@ a_suO)
(@ t_suP)
(w_suQ [Occ=Once] :: [a_suO])
_ [Occ=Dead] ->
Foo.$wf @ a_suO @ t_suP w_suQ}]
f = \ (@ a_suO) (@ t_suP) (w_suQ :: [a_suO]) _ [Occ=Dead] ->
Foo.$wf @ a_suO @ t_suP w_suQ
-- RHS size: {terms: 2, types: 0, coercions: 0, joins: 0/0}
Foo.boo1 :: Char
[GblId,
Caf=NoCafRefs,
Str=m,
Unf=Unf{Src=<vanilla>, TopLvl=True, Value=True, ConLike=True,
WorkFree=True, Expandable=True, Guidance=IF_ARGS [] 10 20}]
Foo.boo1 = GHC.Types.C# 'v'#
-- RHS size: {terms: 6, types: 6, coercions: 0, joins: 0/0}
boo :: forall a. a -> [a] -> Bool
[GblId,
Arity=2,
Caf=NoCafRefs,
Str=<L,A><S,1*U>,
Unf=Unf{Src=InlineStable, TopLvl=True, Value=True, ConLike=True,
WorkFree=True, Expandable=True,
Guidance=ALWAYS_IF(arity=2,unsat_ok=True,boring_ok=False)
Tmpl= \ (@ a_aud)
(p_atA [Occ=Once] :: a_aud)
(ps_atB [Occ=Once] :: [a_aud]) ->
f @ a_aud @ Char (GHC.Types.: @ a_aud p_atA ps_atB) Foo.boo1}]
boo
= \ (@ a_aud) (p_atA :: a_aud) (ps_atB :: [a_aud]) ->
Foo.f_$s$wf @ a_aud p_atA ps_atB
------ Local rules for imported ids --------
"SC:$wf0" [2]
forall (@ a_aud) (sc_sv2 :: a_aud) (sc1_sv3 :: [a_aud]).
Foo.$wf @ a_aud @ Char (GHC.Types.: @ a_aud sc_sv2 sc1_sv3)
= Foo.f_$s$wf @ a_aud sc_sv2 sc1_sv3
```
The worker `$wf` drops `y`, as it should, as a result of demand analysis. But `a` is unused too, and lo! `sc_sv2` is passed by the RULE to `f_$s$wf`, which indeed does not use it.
This is fixed by `-flate-dmd-anal`, but there is a missed opportunity. SpecConstr actually gathers `NoOcc` info, and could exploit it to pass fewer arguments to the specialised function.https://gitlab.haskell.org/ghc/ghc/-/issues/20320Argument is marked as absent when it likely should not be.2022-05-24T14:32:37ZAndreas KlebingerArgument is marked as absent when it likely should not be.This function from nofib/real/ben-raytrace carries an absent demand on the first argument. But neither does it seem absent (it appears in the body) nor should workers take an absent demand to begin with.
I'm a bit puzzled. This is from ...This function from nofib/real/ben-raytrace carries an absent demand on the first argument. But neither does it seem absent (it appears in the body) nor should workers take an absent demand to begin with.
I'm a bit puzzled. This is from a recent build on master.
Here is the core function in full:
<details>
```
Rec {
-- RHS size: {terms: 298, types: 320, coercions: 50, joins: 2/6}
Sampler.sampleImagePixel_$s$wsample [Occ=LoopBreaker]
:: forall {s}.
BoundingBox.BoundingBox
-> (Ray3 -> Interval -> Maybe Hit)
-> GHC.Prim.Int#
-> (Ray3 -> Colour)
-> Pt Vec3
-> Vec3
-> Random.GenState
-> GHC.Prim.State# s
-> (# GHC.Prim.State# s, (Colour, Random.GenState) #)
[GblId,
Arity=8,
Str= <A>
<LCL(C1(L))>
<1L>
<MCM(L)>
<L>
<L>
<L>
<L>,
Unf=OtherCon []]
Sampler.sampleImagePixel_$s$wsample
= \ (@s_s2m1)
(sc_s2pI :: BoundingBox.BoundingBox)
(sc1_s2pJ :: Ray3 -> Interval -> Maybe Hit)
(sc2_s2pH :: GHC.Prim.Int#)
(sc3_s2pG :: Ray3 -> Colour)
(ww_s2ma
:: Pt Vec3
Unf=OtherCon [])
(ww1_s2mb
:: Vec3
Unf=OtherCon [])
(eta_s2md [OS=OneShot] :: Random.GenState)
(eta1_s2me [OS=OneShot] :: GHC.Prim.State# s_s2m1)
->
case sc2_s2pH of ds_X2 {
__DEFAULT ->
let {
ray_X3 :: Ray3
[LclId, Unf=OtherCon []]
ray_X3 = Ray.Ray3 ww_s2ma ww1_s2mb } in
case sc1_s2pJ ray_X3 lvl3_r2rg of {
Nothing -> (# eta1_s2me, (sc3_s2pG ray_X3, eta_s2md) #);
Just hit_aRy ->
case hit_aRy of wild1_a1W5
{ Hit bx_a1Wc ds1_a1Wd ds2_a1We ds3_a1Wf ds4_a1Wg ->
case ds4_a1Wg of { Material ds7_s2of ds8_s2og ->
case ((((ds7_s2of @s_s2m1 ray_X3 wild1_a1W5)
`cast` (SamplerMonad.Naive.N:SamplerM[0] <s_s2m1>_N <(Vec3,
Maybe Ray3)>_N
; Control.Monad.Trans.State.Strict.N:StateT[0]
<System.Random.Internal.StdGen>_N
<GHC.ST.ST s_s2m1>_R
<(Vec3, Maybe Ray3)>_N
:: SamplerM s_s2m1 (Vec3, Maybe Ray3)
~R# (System.Random.Internal.StdGen
-> GHC.ST.ST
s_s2m1
((Vec3, Maybe Ray3), System.Random.Internal.StdGen))))
eta_s2md)
`cast` (GHC.ST.N:ST[0]
<s_s2m1>_N <((Vec3, Maybe Ray3), System.Random.Internal.StdGen)>_R
:: GHC.ST.ST
s_s2m1 ((Vec3, Maybe Ray3), System.Random.Internal.StdGen)
~R# GHC.ST.STRep
s_s2m1 ((Vec3, Maybe Ray3), System.Random.Internal.StdGen)))
eta1_s2me
of
{ (# ipv_a29P, ipv1_a29Q #) ->
case ipv1_a29Q of { (a1_a29T, s'_a29U) ->
case a1_a29T of { (atten_aRA, mb_scattered_aRB) ->
case mb_scattered_aRB of {
Nothing ->
case ds8_s2og of {
Figure.NoEmission ->
(# ipv_a29P,
(Colour.black1
`cast` (Sym (Colour.N:Colour[0]) :: Vec3 ~R# Colour),
s'_a29U) #);
Figure.ConstEmission x_a1Vb -> (# ipv_a29P, (x_a1Vb, s'_a29U) #)
};
Just scattered_aRC ->
case GHC.Prim.<# ds_X2 45# of {
__DEFAULT ->
case scattered_aRC of { Ray3 ww2_X9 ww3_Xa ->
case Sampler.sampleImagePixel_$s$wsample
@s_s2m1
sc_s2pI
sc1_s2pJ
(GHC.Prim.-# ds_X2 1#)
sc3_s2pG
ww2_X9
ww3_Xa
s'_a29U
ipv_a29P
of
{ (# ipv2_Xc, ipv3_Xd #) ->
case ipv3_Xd of { (a2_Xf, s'1_Xg) ->
case ds8_s2og of {
Figure.NoEmission ->
case atten_aRA of { Vec3 bx1_a2aF bx2_a2aG bx3_a2aH ->
case a2_Xf `cast` (Colour.N:Colour[0] :: Colour ~R# Vec3) of
{ Vec3 bx4_a2aK bx5_a2aL bx6_a2aM ->
(# ipv2_Xc,
((Vector.Vec3
(GHC.Prim.*## bx1_a2aF bx4_a2aK)
(GHC.Prim.*## bx2_a2aG bx5_a2aL)
(GHC.Prim.*## bx3_a2aH bx6_a2aM))
`cast` (Sym (Colour.N:Colour[0]) :: Vec3 ~R# Colour),
s'1_Xg) #)
}
};
Figure.ConstEmission x_a1Vb ->
case x_a1Vb `cast` (Colour.N:Colour[0] :: Colour ~R# Vec3) of
{ Vec3 bx3_s2oj bx4_s2ok bx5_s2ol ->
case atten_aRA of { Vec3 bx1_a2aF bx2_a2aG bx6_a2aH ->
case a2_Xf `cast` (Colour.N:Colour[0] :: Colour ~R# Vec3) of
{ Vec3 bx7_a2aK bx8_a2aL bx9_a2aM ->
(# ipv2_Xc,
((Vector.Vec3
(GHC.Prim.+## (GHC.Prim.*## bx1_a2aF bx7_a2aK) bx3_s2oj)
(GHC.Prim.+## (GHC.Prim.*## bx2_a2aG bx8_a2aL) bx4_s2ok)
(GHC.Prim.+## (GHC.Prim.*## bx6_a2aH bx9_a2aM) bx5_s2ol))
`cast` (Sym (Colour.N:Colour[0]) :: Vec3 ~R# Colour),
s'1_Xg) #)
}
}
}
}
}
}
};
1# ->
case s'_a29U
`cast` (System.Random.Internal.N:StdGen[0]
:: System.Random.Internal.StdGen
~R# System.Random.SplitMix.SMGen)
of
{ System.Random.SplitMix.SMGen bx1_a2jl bx2_a2jm ->
case atten_aRA of { Vec3 bx3_a1VB bx4_a1VC bx5_a1VD ->
join {
$w$j_s2lZ [InlPrag=[2], Dmd=1C1(L)]
:: GHC.Prim.Double#
-> (# GHC.Prim.State# s_s2m1, (Colour, Random.GenState) #)
[LclId[JoinId(1)], Arity=1, Str=<L>, Unf=OtherCon []]
$w$j_s2lZ (x1_s2lW [OS=OneShot] :: GHC.Prim.Double#)
= join {
$j_s2jH [Dmd=1C1(L)]
:: GHC.Prim.Double#
-> (# GHC.Prim.State# s_s2m1, (Colour, Random.GenState) #)
[LclId[JoinId(1)], Arity=1, Str=<L>, Unf=OtherCon []]
$j_s2jH (x_a2b7 [OS=OneShot] :: GHC.Prim.Double#)
= let {
seed'_a2jk :: GHC.Prim.Word#
[LclId]
seed'_a2jk = GHC.Prim.plusWord# bx1_a2jl bx2_a2jm } in
let {
x#_a2jo :: GHC.Prim.Word#
[LclId]
x#_a2jo
= GHC.Prim.timesWord#
(GHC.Prim.xor#
seed'_a2jk (GHC.Prim.uncheckedShiftRL# seed'_a2jk 33#))
18397679294719823053## } in
let {
x#1_a2jp :: GHC.Prim.Word#
[LclId]
x#1_a2jp
= GHC.Prim.timesWord#
(GHC.Prim.xor#
x#_a2jo (GHC.Prim.uncheckedShiftRL# x#_a2jo 33#))
14181476777654086739## } in
case GHC.Prim.>##
x_a2b7
(GHC.Prim./##
(GHC.Prim.word2Double#
(GHC.Prim.xor#
x#1_a2jp
(GHC.Prim.uncheckedShiftRL# x#1_a2jp 33#)))
1.8446744073709552e19##)
of {
__DEFAULT ->
case ds8_s2og of {
Figure.NoEmission ->
(# ipv_a29P,
(Colour.black1
`cast` (Sym (Colour.N:Colour[0]) :: Vec3 ~R# Colour),
(System.Random.SplitMix.SMGen seed'_a2jk bx2_a2jm)
`cast` (Sym (System.Random.Internal.N:StdGen[0])
:: System.Random.SplitMix.SMGen
~R# System.Random.Internal.StdGen)) #);
Figure.ConstEmission x2_a1Vb ->
(# ipv_a29P,
(x2_a1Vb,
(System.Random.SplitMix.SMGen seed'_a2jk bx2_a2jm)
`cast` (Sym (System.Random.Internal.N:StdGen[0])
:: System.Random.SplitMix.SMGen
~R# System.Random.Internal.StdGen)) #)
};
1# ->
case scattered_aRC of { Ray3 ww2_X9 ww3_Xa ->
case Sampler.sampleImagePixel_$s$wsample
@s_s2m1
sc_s2pI
sc1_s2pJ
(GHC.Prim.-# ds_X2 1#)
sc3_s2pG
ww2_X9
ww3_Xa
((System.Random.SplitMix.SMGen seed'_a2jk bx2_a2jm)
`cast` (Sym (System.Random.Internal.N:StdGen[0])
:: System.Random.SplitMix.SMGen
~R# System.Random.Internal.StdGen))
ipv_a29P
of
{ (# ipv2_Xk, ipv3_Xl #) ->
case ipv3_Xl of { (a2_Xn, s'1_Xo) ->
case ds8_s2og of {
Figure.NoEmission ->
case a2_Xn `cast` (Colour.N:Colour[0] :: Colour ~R# Vec3)
of
{ Vec3 bx6_a2aK bx7_a2aL bx8_a2aM ->
(# ipv2_Xk,
((Vector.Vec3
(GHC.Prim.*## bx3_a1VB bx6_a2aK)
(GHC.Prim.*## bx4_a1VC bx7_a2aL)
(GHC.Prim.*## bx5_a1VD bx8_a2aM))
`cast` (Sym (Colour.N:Colour[0]) :: Vec3 ~R# Colour),
s'1_Xo) #)
};
Figure.ConstEmission x2_a1Vb ->
case x2_a1Vb
`cast` (Colour.N:Colour[0] :: Colour ~R# Vec3)
of
{ Vec3 bx6_s2oo bx7_s2op bx8_s2oq ->
case a2_Xn `cast` (Colour.N:Colour[0] :: Colour ~R# Vec3)
of
{ Vec3 bx9_a2aK bx10_a2aL bx11_a2aM ->
(# ipv2_Xk,
((Vector.Vec3
(GHC.Prim.+##
(GHC.Prim.*## bx3_a1VB bx9_a2aK) bx6_s2oo)
(GHC.Prim.+##
(GHC.Prim.*## bx4_a1VC bx10_a2aL) bx7_s2op)
(GHC.Prim.+##
(GHC.Prim.*## bx5_a1VD bx11_a2aM) bx8_s2oq))
`cast` (Sym (Colour.N:Colour[0]) :: Vec3 ~R# Colour),
s'1_Xo) #)
}
}
}
}
}
}
} } in
case GHC.Prim.<=## x1_s2lW bx5_a1VD of {
__DEFAULT -> jump $j_s2jH x1_s2lW;
1# -> jump $j_s2jH bx5_a1VD
} } in
case GHC.Prim.<=## bx3_a1VB bx4_a1VC of {
__DEFAULT -> jump $w$j_s2lZ bx3_a1VB;
1# -> jump $w$j_s2lZ bx4_a1VC
}
}
}
}
}
}
}
}
}
}
};
-1# ->
(# eta1_s2me,
(Colour.black1
`cast` (Sym (Colour.N:Colour[0]) :: Vec3 ~R# Colour),
eta_s2md) #)
}
end Rec }
```
</details>
@sgraf812 Maybe you have an idea?https://gitlab.haskell.org/ghc/ghc/-/issues/20321SpecConstr increasing allocation. Maybe by triggering reboxing2022-05-17T14:59:44ZAndreas KlebingerSpecConstr increasing allocation. Maybe by triggering reboxingI noticed that nofib/real/ben-raytrace seems to increase allocations quite significantly with -fspec-constr compared to without.
In nofib/real/ben-raytracer we have this function:
```haskell
hitTestMany :: [Figure] -> Ray3 -> Interval ...I noticed that nofib/real/ben-raytrace seems to increase allocations quite significantly with -fspec-constr compared to without.
In nofib/real/ben-raytracer we have this function:
```haskell
hitTestMany :: [Figure] -> Ray3 -> Interval -> Maybe Hit
hitTestMany figs ray = go_hit figs Nothing
where
go_hit :: [Figure] -> Maybe Hit -> Interval -> Maybe Hit
go_hit [] accum !_ = accum
go_hit (fig:rest) accum int =
case hitTest fig ray int of
Nothing -> go_hit rest accum int
Just hit ->
case accum of
Nothing
-> go_hit rest (Just hit) (int {iUpper=hitDistance hit})
Just oldHit
| hitDistance hit < hitDistance oldHit
-> go_hit rest (Just hit) (int {iUpper=hitDistance hit})
| otherwise
-> go_hit rest accum int
```
Without spec-constr we get somewhat straight forward core:
```haskell
joinrec {
$wgo_hit_s2L4 [InlPrag=[2],
Occ=LoopBreaker,
Dmd=SCS(C1(C1(C1(L))))]
:: [Figure]
-> Maybe Hit -> GHC.Prim.Double# -> GHC.Prim.Double# -> Maybe Hit
[LclId[JoinId(4)], Arity=4, Str=<1L><1L><L><L>, Unf=OtherCon []]
$wgo_hit_s2L4 (ds_s2KW :: [Figure])
(accum_s2KX :: Maybe Hit)
(ww9_X2 :: GHC.Prim.Double#)
(ww10_X3 :: GHC.Prim.Double#)
= case ds_s2KW of {
[] -> accum_s2KX;
: fig_aSB rest_aSC ->
case fig_aSB of { Figure ds1_d27m ds2_d27n ->
case ds2_d27n ray_aSy (Interval.Interval ww9_X2 ww10_X3)
of wild4_X6 {
Nothing -> jump $wgo_hit_s2L4 rest_aSC accum_s2KX ww9_X2 ww10_X3;
Just hit_aSF ->
case accum_s2KX of wild5_X7 {
Nothing ->
case hit_aSF of
{ Hit bx_d2f3 ds3_d27x ds4_d27y ds5_d27z ds6_d27A ->
jump $wgo_hit_s2L4 rest_aSC wild4_X6 ww9_X2 bx_d2f3
};
Just oldHit_aSG ->
case hit_aSF of
{ Hit bx_d2f3 ds3_d27x ds4_d27y ds5_d27z ds6_d27A ->
case oldHit_aSG of { Hit bx1_Xa ds7_Xb ds8_Xc ds9_Xd ds10_Xe ->
case GHC.Prim.<## bx_d2f3 bx1_Xa of {
__DEFAULT -> jump $wgo_hit_s2L4 rest_aSC wild5_X7 ww9_X2 ww10_X3;
1# -> jump $wgo_hit_s2L4 rest_aSC wild4_X6 ww9_X2 bx_d2f3
}
}
}
}
}
}
}; } in
```
Not that we case on the accumulator `accum_s2KX` but pass it along boxed and as-is.
However if we enable SpecConstr instead then we get the code hidden below the spoiler tag:
<details>
```haskell
joinrec {
$s$wgo_hit_s2Pr [Occ=LoopBreaker,
Dmd=LCL(C1(C1(C1(C1(C1(C1(C1(L))))))))]
:: GHC.Prim.Double#
-> GHC.Prim.Double#
-> GHC.Prim.Double#
-> Pt Vec3
-> Vec3
-> Pt Vec2
-> Material
-> [Figure]
-> Maybe Hit
[LclId[JoinId(8)],
Arity=8,
Str=<L><L><L><L><L><L><L><1L>,
Unf=OtherCon []]
$s$wgo_hit_s2Pr (sc_s2Pp :: GHC.Prim.Double#)
(sc1_s2Po :: GHC.Prim.Double#)
(sc2_s2Pj :: GHC.Prim.Double#)
(sc3_s2Pk :: Pt Vec3)
(sc4_s2Pl :: Vec3)
(sc5_s2Pm :: Pt Vec2)
(sc6_s2Pn :: Material)
(sc7_s2Pi :: [Figure])
= case sc7_s2Pi of {
[] ->
GHC.Maybe.Just
@Hit (Figure.Hit sc2_s2Pj sc3_s2Pk sc4_s2Pl sc5_s2Pm sc6_s2Pn);
: fig_aSC rest_aSD ->
case fig_aSC of { Figure ds_d27n ds1_d27o ->
case ds1_d27o ray_aSz (Interval.Interval sc1_s2Po sc_s2Pp) of {
Nothing ->
jump $s$wgo_hit_s2Pr
sc_s2Pp
sc1_s2Po
sc2_s2Pj
sc3_s2Pk
sc4_s2Pl
sc5_s2Pm
sc6_s2Pn
rest_aSD;
Just hit_aSG ->
case hit_aSG of
{ Hit bx_d2f4 ds2_d27y ds3_d27z ds4_d27A ds5_d27B ->
case GHC.Prim.<## bx_d2f4 sc2_s2Pj of {
__DEFAULT ->
jump $s$wgo_hit_s2Pr
sc_s2Pp
sc1_s2Po
sc2_s2Pj
sc3_s2Pk
sc4_s2Pl
sc5_s2Pm
sc6_s2Pn
rest_aSD;
1# ->
jump $s$wgo_hit_s2Pr
bx_d2f4
sc1_s2Po
bx_d2f4
ds2_d27y
ds3_d27z
ds4_d27A
ds5_d27B
rest_aSD
}
}
}
}
}; } in
joinrec {
$s$wgo_hit1_s2Pq [Occ=LoopBreaker, Dmd=LCL(C1(C1(L)))]
:: GHC.Prim.Double# -> GHC.Prim.Double# -> [Figure] -> Maybe Hit
[LclId[JoinId(3)], Arity=3, Str=<L><L><1L>, Unf=OtherCon []]
$s$wgo_hit1_s2Pq (sc_s2Ph :: GHC.Prim.Double#)
(sc1_s2Pg :: GHC.Prim.Double#)
(sc2_s2Pf :: [Figure])
= case sc2_s2Pf of {
[] -> GHC.Maybe.Nothing @Hit;
: fig_aSC rest_aSD ->
case fig_aSC of { Figure ds_d27n ds1_d27o ->
case ds1_d27o ray_aSz (Interval.Interval sc1_s2Pg sc_s2Ph) of {
Nothing -> jump $s$wgo_hit1_s2Pq sc_s2Ph sc1_s2Pg rest_aSD;
Just hit_aSG ->
case hit_aSG of
{ Hit bx_d2f4 ds2_d27y ds3_d27z ds4_d27A ds5_d27B ->
jump $s$wgo_hit_s2Pr
bx_d2f4
sc1_s2Pg
bx_d2f4
ds2_d27y
ds3_d27z
ds4_d27A
ds5_d27B
rest_aSD
}
}
}
}; } in
case figs_s2L7 of { :| a1_a28j as_a28k ->
case a1_a28j of { Figure ds_d27n ds1_d27o ->
case ds1_d27o ray_aSz wild1_X1 of {
Nothing -> jump $s$wgo_hit1_s2Pq ww8_s2L2 ww7_s2L1 figs1_s2CL;
Just hit_aSG ->
case hit_aSG of
{ Hit bx_d2f4 ds2_d27y ds3_d27z ds4_d27A ds5_d27B ->
jump $s$wgo_hit_s2Pr
bx_d2f4
ww7_s2L1
bx_d2f4
ds2_d27y
ds3_d27z
ds4_d27A
ds5_d27B
figs1_s2CL
}
}
```
</details>
SpecConstr seems to specialise on the accumulator and in the `Just` case also unboxes the Hit constructor.
However we eventually want to return the boxed `Just (Hit ...)` constructor just the way we got it. So we eventually end up reboxing it (and the Just it's contained in).
This is quite annoying. I guess this could eventually be tied into the boxity analysis @sgraf812 mentions in #19871.Research neededhttps://gitlab.haskell.org/ghc/ghc/-/issues/21456SpecConstr: -fspec-constr-threshold has likely some undesired behaviour2024-01-29T15:44:15ZAndreas KlebingerSpecConstr: -fspec-constr-threshold has likely some undesired behaviourAs the user guide helpfully points out `-fspec-constr-threshold` does "Set the size threshold for the SpecConstr transformation."
Currently best I can tell what it does is that if any rhss of a recursive group is larger than `threshold`...As the user guide helpfully points out `-fspec-constr-threshold` does "Set the size threshold for the SpecConstr transformation."
Currently best I can tell what it does is that if any rhss of a recursive group is larger than `threshold` we avoid specializing any of them. I assume to avoid code blowup. So far so good.
----
But I think we still traverse the AST looking for local bindings to specialize. We don't check the size of local bindings when deciding if we should specialise. So if the top level binding is just a thin (recursive) function over a huge local binding we would still end up specializing the local binding anyway.
As a result using `-fspec-constr-threshold` avoid code size blowups is hit and miss.
We also make the decision for the whole recursive group. But if we have some group:
```
Rec {
f_huge = <huge>
f_2 = <smallish>
f_3 = <smallish>
}
```
We should still specialize `f_2` and `f_3` for all call patterns inside the whole group. There is really no good reason not to!https://gitlab.haskell.org/ghc/ghc/-/issues/21457SpecConstr should treat local and top level non-recursive bindings the same.2023-11-10T12:10:59ZAndreas KlebingerSpecConstr should treat local and top level non-recursive bindings the same.SpecConstr currently specializes local non recursive let bindings. But doesn't specialize top level non-recursive let bindings.
This means if we change the local binding in a way which allows it to float to the top performance can sudde...SpecConstr currently specializes local non recursive let bindings. But doesn't specialize top level non-recursive let bindings.
This means if we change the local binding in a way which allows it to float to the top performance can suddenly tank as we no longer get specialization for it.
--------------
This is however not as trivial as it sounds. In order to make this work coherently in the current design we would need to analyze the whole module for call patterns before we generate a specialization.
But we analyze the whole module for call patterns already! So maybe we could get away with:
* Do what we do now but keep around all call patterns for non-recursive top level bindings.
* We collected all the call patterns and specialize all the recursive bindings.
* Then do *one* pass over the non-recursive bindings generating specializations for their call patterns as well.
Seems plausible at least.