GHC issueshttps://gitlab.haskell.org/ghc/ghc/-/issues2023-09-07T10:56:34Zhttps://gitlab.haskell.org/ghc/ghc/-/issues/22972Providing IPE-enabled bindists in the release pipeline2023-09-07T10:56:34ZHécate MoonlightProviding IPE-enabled bindists in the release pipelineIt would significantly improve the usability of info-table profiling if IPE-enabled bindists were provided by our release pipeline.It would significantly improve the usability of info-table profiling if IPE-enabled bindists were provided by our release pipeline.9.10.1Matthew PickeringMatthew Pickeringhttps://gitlab.haskell.org/ghc/ghc/-/issues/22473Make `wrapBox` work for AddrRep, TupleRep, SumRep, VecRep2023-06-19T11:55:26ZSebastian GrafMake `wrapBox` work for AddrRep, TupleRep, SumRep, VecRep!8750 beefed up our ability to allocate a box for many different representations.
Notable exceptions are `AddrRep`, `TupleRep`, `SumRep` and `VecRep`.
#22336 suggests to give up with a good error message when encountering these repres...!8750 beefed up our ability to allocate a box for many different representations.
Notable exceptions are `AddrRep`, `TupleRep`, `SumRep` and `VecRep`.
#22336 suggests to give up with a good error message when encountering these representations. This ticket is more ambitious: It asks for boxes for all these representations. This would be tremendously helpful for #17521 and #20269.
But since `TupleRep` and `SumRep` are recursive, there are an inifinite number of different representations to cover. Hence we can never catch all of them with a finite number of `*Box` data types as has been begun in `ghc-prim:GHC.Types`.
# Proposal
1. Define a wired-in type or data family
```hs
type Box :: forall {r :: RuntimeRep}. TYPE r -> Type
data family Box @r a
```
Some instances of this family could be defined to delegate to the existing `*Box` data types in `ghc-prim:GHC.Types`, but the hard problem remains with unboxed tuples. This is the tricky part! We'll discuss the design below.
!9353 proposes to create a data family instance on the fly, at representation `Rep`. The representation `TyCon` is `R:Box[Rep]`, with a single data constructor `MkBox[Rep] :: forall (a :: TYPE Rep). a -> R:Box[Rep]` and coercion `co = D:R:Box[Rep] <ty>_N :: Box @Rep ty ~R# R:Box[Rep] ty`. (Side note: this data family has roles `[Nominal, Representational]`, whereas usually only `Nominal` roles are allowed in data families. This allows coercing between `Box a` and `Box b` when `a` and `b` are representationally equal.) (Kindly stole this explanation from @sheaf.)
I want to add that we already have matching heap objects, if I understand `CONSTR` correctly. Maybe we could also pick `CONSTR_STATIC` when applicable. And of course we can optimise to `CONSTR_p_n` if we have a fitting combination for the given `r` (which will be 99% of cases). So I think we might not even need to generate an excessive amount of info tables for the special `MkBox` DataCons.
Having `Box` would resolve #20269.
2. #17521 also wants to offer an API to the user so that unlifted bindings at the top-level are possible (and memoised). One could imagine special, wired-in functions
```hs
box :: forall {r :: RuntimeRep} (a :: TYPE r). a -> Box a
unbox :: forall {r :: RuntimeRep} (a :: TYPE r). Box a -> a
```
which basically desugar to something that GHC could resolve with `GHC.Core.wrapBox`. It might be possible to put them in a type class `Boxing`, but I don't yet see the added value of doing so.
Note that (1) can be done independently of (2), but I think it makes sense to do them together.
The instances for `Box` might re-use many of the existing `*Box` types in `ghc-prim:GHC.Types` or might not. Not sure. But upon further reflection, I don't think !8750 brought us any closer to a satisfying resolution of either #20269 or #17521.https://gitlab.haskell.org/ghc/ghc/-/issues/14844SpecConstr also non-recursive top-level functions2022-10-17T16:48:21ZJoachim Breitnermail@joachim-breitner.deSpecConstr also non-recursive top-level functionsIn order to fix the regressions introduced by loopification (#14068), we probably need to get !SpecConstr also do something about non-recursive functions.
In a first iteration, we might want to enable it for non-recursive functions stra...In order to fix the regressions introduced by loopification (#14068), we probably need to get !SpecConstr also do something about non-recursive functions.
In a first iteration, we might want to enable it for non-recursive functions straight-away, and see if it fixes the regressions introduced by Loopification.
But if this turns out to be too slow/too expensive, we should try specializing non-recursive *local* functions *if there is only one call-patterns*. That ought to be a straight win in all cases anyways.
<details><summary>Trac metadata</summary>
| Trac field | Value |
| ---------------------- | ------------ |
| Version | 8.5 |
| Type | Task |
| TypeOfFailure | OtherFailure |
| Priority | normal |
| Resolution | Unresolved |
| Component | Compiler |
| Test case | |
| Differential revisions | |
| BlockedBy | |
| Related | |
| Blocking | |
| CC | |
| Operating system | |
| Architecture | |
</details>
<!-- {"blocked_by":[],"summary":"SpecConstr also non-recursive function","status":"New","operating_system":"","component":"Compiler","related":[],"milestone":"","resolution":"Unresolved","owner":{"tag":"Unowned"},"version":"8.5","keywords":[],"differentials":[],"test_case":"","architecture":"","cc":[""],"type":"Task","description":"In order to fix the regressions introduced by loopification (#14068), we probably need to get !SpecConstr also do something about non-recursive functions.\r\n\r\nIn a first iteration, we might want to enable it for non-recursive functions straight-away, and see if it fixes the regressions introduced by Loopification.\r\n\r\nBut if this turns out to be too slow/too expensive, we should try specializing non-recursive ''local'' functions ''if there is only one call-patterns''. That ought to be a straight win in all cases anyways.\r\n\r\n\r\n\r\n","type_of_failure":"OtherFailure","blocking":[]} -->https://gitlab.haskell.org/ghc/ghc/-/issues/14816DmdAnal punishes free variables compared to static argument encoding2022-11-30T20:18:40ZDavid FeuerDmdAnal punishes free variables compared to static argument encodingWhen I compile
```hs
test :: (a -> a -> a) -> Int -> a -> HashMap Int a -> HashMap Int a
test f k a m = insertModifying a (blink (f a)) k m
blink :: (a -> b) -> a -> (# b #)
-- Or blink g = \a -> (# g a #) ; it makes no difference.
bli...When I compile
```hs
test :: (a -> a -> a) -> Int -> a -> HashMap Int a -> HashMap Int a
test f k a m = insertModifying a (blink (f a)) k m
blink :: (a -> b) -> a -> (# b #)
-- Or blink g = \a -> (# g a #) ; it makes no difference.
blink g a = (# g a #)
```
I get
```
test
= \ (@ a_a9o3)
(f_a7iP :: a_a9o3 -> a_a9o3 -> a_a9o3)
(k_a7iQ :: Int)
(a1_a7iR :: a_a9o3)
(m_a7iS :: HashMap Int a_a9o3) ->
case k_a7iQ of { GHC.Types.I# ww1_sa1Z ->
RULES.$w$sinsertModifying
@ a_a9o3
a1_a7iR
(let {
g_s9E1 [Dmd=<L,C(U)>] :: a_a9o3 -> a_a9o3
[LclId]
g_s9E1 = f_a7iP a1_a7iR } in
\ (a2_a7iU :: a_a9o3) -> (# g_s9E1 a2_a7iU #))
ww1_sa1Z
m_a7iS
}
```
We build `g_s9E1 = f_a7iP a1_a7iR` for no apparent reason. Trouble persists into STG:
```
RULES.test
:: forall a.
(a -> a -> a)
-> GHC.Types.Int
-> a
-> Data.HashMap.Base.HashMap GHC.Types.Int a
-> Data.HashMap.Base.HashMap GHC.Types.Int a
[GblId,
Arity=4,
Str=<L,1*C1(C(U))><S(S),1*U(U)><L,U><S,1*U>,
Unf=OtherCon []] =
[] \r [f_saiX k_saiY a1_saiZ m_saj0]
case k_saiY of {
GHC.Types.I# ww1_saj2 [Occ=Once] ->
let {
g_saj3 [Occ=OnceL!, Dmd=<L,C(U)>] :: a_a9o3 -> a_a9o3
[LclId] =
[f_saiX a1_saiZ] \u [] f_saiX a1_saiZ; } in
let {
sat_saj6 [Occ=Once] :: a_a9o3 -> (# a_a9o3 #)
[LclId] =
[g_saj3] \r [a2_saj4]
let {
sat_saj5 [Occ=Once] :: a_a9o3
[LclId] =
[g_saj3 a2_saj4] \u [] g_saj3 a2_saj4;
} in Unit# [sat_saj5];
} in RULES.$w$sinsertModifying a1_saiZ sat_saj6 ww1_saj2 m_saj0;
};
```
`insertModifying` uses its function argument at most once, so there is no possible benefit to this partial application.
<details><summary>Trac metadata</summary>
| Trac field | Value |
| ---------------------- | ------------ |
| Version | 8.2.2 |
| Type | Bug |
| TypeOfFailure | OtherFailure |
| Priority | normal |
| Resolution | Unresolved |
| Component | Compiler |
| Test case | |
| Differential revisions | |
| BlockedBy | |
| Related | |
| Blocking | |
| CC | nomeata |
| Operating system | |
| Architecture | |
</details>
<!-- {"blocked_by":[],"summary":"Missed Called Arity opportunity?","status":"New","operating_system":"","component":"Compiler","related":[],"milestone":"8.6.1","resolution":"Unresolved","owner":{"tag":"Unowned"},"version":"8.2.2","keywords":[],"differentials":[],"test_case":"","architecture":"","cc":["nomeata"],"type":"Bug","description":"When I compile\r\n\r\n{{{#!hs\r\ntest :: (a -> a -> a) -> Int -> a -> HashMap Int a -> HashMap Int a\r\ntest f k a m = insertModifying a (blink (f a)) k m\r\n\r\nblink :: (a -> b) -> a -> (# b #)\r\n-- Or blink g = \\a -> (# g a #) ; it makes no difference.\r\nblink g a = (# g a #)\r\n}}}\r\n\r\nI get\r\n\r\n{{{\r\ntest\r\n = \\ (@ a_a9o3)\r\n (f_a7iP :: a_a9o3 -> a_a9o3 -> a_a9o3)\r\n (k_a7iQ :: Int)\r\n (a1_a7iR :: a_a9o3)\r\n (m_a7iS :: HashMap Int a_a9o3) ->\r\n case k_a7iQ of { GHC.Types.I# ww1_sa1Z ->\r\n RULES.$w$sinsertModifying\r\n @ a_a9o3\r\n a1_a7iR\r\n (let {\r\n g_s9E1 [Dmd=<L,C(U)>] :: a_a9o3 -> a_a9o3\r\n [LclId]\r\n g_s9E1 = f_a7iP a1_a7iR } in\r\n \\ (a2_a7iU :: a_a9o3) -> (# g_s9E1 a2_a7iU #))\r\n ww1_sa1Z\r\n m_a7iS\r\n }\r\n}}}\r\n\r\nWe build `g_s9E1 = f_a7iP a1_a7iR` for no apparent reason. Trouble persists into STG:\r\n\r\n{{{\r\nRULES.test\r\n :: forall a.\r\n (a -> a -> a)\r\n -> GHC.Types.Int\r\n -> a\r\n -> Data.HashMap.Base.HashMap GHC.Types.Int a\r\n -> Data.HashMap.Base.HashMap GHC.Types.Int a\r\n[GblId,\r\n Arity=4,\r\n Str=<L,1*C1(C(U))><S(S),1*U(U)><L,U><S,1*U>,\r\n Unf=OtherCon []] =\r\n [] \\r [f_saiX k_saiY a1_saiZ m_saj0]\r\n case k_saiY of {\r\n GHC.Types.I# ww1_saj2 [Occ=Once] ->\r\n let {\r\n g_saj3 [Occ=OnceL!, Dmd=<L,C(U)>] :: a_a9o3 -> a_a9o3\r\n [LclId] =\r\n [f_saiX a1_saiZ] \\u [] f_saiX a1_saiZ; } in\r\n let {\r\n sat_saj6 [Occ=Once] :: a_a9o3 -> (# a_a9o3 #)\r\n [LclId] =\r\n [g_saj3] \\r [a2_saj4]\r\n let {\r\n sat_saj5 [Occ=Once] :: a_a9o3\r\n [LclId] =\r\n [g_saj3 a2_saj4] \\u [] g_saj3 a2_saj4;\r\n } in Unit# [sat_saj5];\r\n } in RULES.$w$sinsertModifying a1_saiZ sat_saj6 ww1_saj2 m_saj0;\r\n };\r\n}}}\r\n\r\n`insertModifying` uses its function argument at most once, so there is no possible benefit to this partial application.","type_of_failure":"OtherFailure","blocking":[]} -->https://gitlab.haskell.org/ghc/ghc/-/issues/24568Make ghc-toolchain vs configure differences errors instead of warnings2024-03-26T15:35:28ZRodrigo MesquitaMake ghc-toolchain vs configure differences errors instead of warningsI believe ghc-toolchain has been introduced for over 2 releases now, and that we should proceed to the next stage: making it the default and deleting toolchain configuration logic from configure.
To this end, we should try to make the c...I believe ghc-toolchain has been introduced for over 2 releases now, and that we should proceed to the next stage: making it the default and deleting toolchain configuration logic from configure.
To this end, we should try to make the current configure warning which diffs the outputs of configure and ghc-toolchain into errors. Given the lack of recent reports about toolchains differences and the fact that CI currently checks for these differences as errors, they should not be terribly common and push remaining problems to be reported...
Although this may be too heavy of a hammer. It would be good to discuss what should be the path forward for a ghc-toolchain-only-future with high assurance.9.14.1https://gitlab.haskell.org/ghc/ghc/-/issues/24419GHCi statements to use HsExpansions route2024-02-15T15:43:46ZApoorv IngleGHCi statements to use HsExpansions route## Summary
Currently GHCi statements are typechecked using `tcStmtsAndThen` in `GHC.Tc.Gen.TcGhciStmts`.
`tcStmtsAndThen` internally uses `tcSyntaxOp` to typecheck statements.
We should instead be using `GHC.Tc.Gen.Do.expandDoStmts` ...## Summary
Currently GHCi statements are typechecked using `tcStmtsAndThen` in `GHC.Tc.Gen.TcGhciStmts`.
`tcStmtsAndThen` internally uses `tcSyntaxOp` to typecheck statements.
We should instead be using `GHC.Tc.Gen.Do.expandDoStmts` so that we can start removing the dependency from `tcSyntaxOp`
## Steps to reproduce
ghci script
```haskell
:set -XImpredicativeTypes
type Id = forall a. a -> a
t :: IO Id; t = return id
p :: Id -> (Bool, Int); p f = (f True, f 3)
x <- t
```
fails with the following error message:
```
Couldn't match type ‘a0’ with ‘Id’
Expected: IO a0
Actual: IO Id
Cannot instantiate unification variable ‘a0’
with a type involving polytypes: Id
In a stmt of an interactive GHCi command:
x <- GHC.GHCi.ghciStepIO :: IO a -> IO a t
```
Expected output is that it should assign polymorphic typed value `id` to `x`
## GHC
version: 9.9 (Head)Apoorv IngleApoorv Inglehttps://gitlab.haskell.org/ghc/ghc/-/issues/24271CprAnal: Record evaluatedness of result fields2023-12-22T18:06:32ZMatthew Cravenclyring@gmail.comCprAnal: Record evaluatedness of result fieldsCPR analysis already gathers this information, since it is needed for nested CPR. Recording this information would allow the simplifier to use it to drop redundant evals around call sites. (It would also make the special case for `seq#` ...CPR analysis already gathers this information, since it is needed for nested CPR. Recording this information would allow the simplifier to use it to drop redundant evals around call sites. (It would also make the special case for `seq#` added for the same reasons in #15226 not special anymore.)
With !9874 this evaluatedness information will help case-of-known-constructor and the like _create_ fewer redundant evals.
We probably eventually want to use these field-evaluatedness-signatures to improve the evaluatedness detection in CPR analysis, to catch recursive functions like this:
```haskell
f :: Integer -> Bool -> (Integer, Integer)
f !x True = (expensive x, x)
f x False = case f (x `quot` 2) (even x) of (p, q) -> (p + 1, q)
-- $wf :: Integer -> Bool -> (# Integer, Integer #)
```
The second field of `f`'s result is always evaluated, but an induction argument is required to prove it. Tag inference can already detect that the second component of `$wf`'s result will be evaluated and properly tagged, so in order to replace the tag inference analysis with worker/wrapper as suggested in #23890 our result-component-evaluatedness analysis in Core must catch this, too.https://gitlab.haskell.org/ghc/ghc/-/issues/23871Testing ghc-toolchain in packaging environments2024-03-21T09:52:05ZMatthew PickeringTesting ghc-toolchain in packaging environmentsWe want to test that ghc-toolchain works in as many situations as possible. This will require building ghc with the `--enable-strict-ghc-toolchain-check` configure flag and verifying that the output produced by ghc-toolchain is the same ...We want to test that ghc-toolchain works in as many situations as possible. This will require building ghc with the `--enable-strict-ghc-toolchain-check` configure flag and verifying that the output produced by ghc-toolchain is the same as the configure script.
Here are a list of potential places where we will want to test this:
- [x] GHC CI itself ( !10976 )
- [ ] Fedora packaging (cc @juhp)
- [ ] Debian packaging
- [ ] Homebrew packaging
- [ ] Haskell.nix packaging
- [ ] nixpkgs packaging
Perhaps people have some other ideas about places where GHC is built from source we should also be testing.Matthew PickeringMatthew Pickeringhttps://gitlab.haskell.org/ghc/ghc/-/issues/23728ghc-toolchain should output to standard output by default2023-07-31T09:48:41ZRodrigo Mesquitaghc-toolchain should output to standard output by defaultCurrently, ghc-toolchain takes `--output=...` as a *required* flag.
I think it would be cleaner and more in-line with the standard for cli programs if we instead outputted the configured `Target` to the standard out if no output file wa...Currently, ghc-toolchain takes `--output=...` as a *required* flag.
I think it would be cleaner and more in-line with the standard for cli programs if we instead outputted the configured `Target` to the standard out if no output file was specified.
cc @bgamariAndré CostaAndré Costahttps://gitlab.haskell.org/ghc/ghc/-/issues/23646Don't depend upon `touch` executable2023-07-24T17:34:59ZBen GamariDon't depend upon `touch` executableCurrently GHC depends upon the POSIX `touch` command for modifying the mtime of files. This incurs a performance tax on some platforms (e.g. Windows) and, more importantly, a significant amount of complexity since we must:
* configure ...Currently GHC depends upon the POSIX `touch` command for modifying the mtime of files. This incurs a performance tax on some platforms (e.g. Windows) and, more importantly, a significant amount of complexity since we must:
* configure the path to `touch` and propagate this information through the `settings` file
* provide an implementation of `touch` (named `touchy`) on Windows
All of this complexity adds a few hundred lines to GHC which can easily be replaced by a five line Haskell implementation. I suggest that we do so.Ben GamariBen Gamarihttps://gitlab.haskell.org/ghc/ghc/-/issues/23608$tooldir variable in settings should not exist2023-08-23T17:44:01ZMatthew Pickering$tooldir variable in settings should not existAfter a call with @alt-romes we discussed why 5d76846405240c051b00cddcda9d8d02c880968e introduced support for the `$tooldir` variable in a settings file.
It seems that this support for finding the tooldir (see `findToolDir` which looks...After a call with @alt-romes we discussed why 5d76846405240c051b00cddcda9d8d02c880968e introduced support for the `$tooldir` variable in a settings file.
It seems that this support for finding the tooldir (see `findToolDir` which looks upwards two directories) was implemented as workaround for the fact that the bundled mingw toolchain exists in two diffent places during a build and in the bindist.
1. During a build `mingw` exists at `$topdir/../../mingw` because the bundled toolchain is placed in `_build/mingw` rather than `_build/stageN/mingw`, and topdir is `_build/stageN/lib`.
2. In a bindist the `mingw` folder is in `$topdir/../mingw`
It seems that a better solution is to have a different settings file for build time and install time
* The settings file during a build points to `$topdir/../../mingw`
* The settings file in the bindist points to `$topdir/../mingw`
This situation mirrors the same situation as with linux builds where we have one settings file which is used during the build process and then a generate a different settings file at install time. The difference is that on windows we can generate this "install time" settings file at build time because it's the same for all installations.
cc @alt-romeshttps://gitlab.haskell.org/ghc/ghc/-/issues/23150Making INLINE functions behave more like normal functions2023-12-14T16:41:26ZSimon Peyton JonesMaking INLINE functions behave more like normal functionsSee also
* #23293
* #22886
* #23931
* #23584 (refactor INLINE pragmas)
If you have
```
f x y z = <expr>
h xs = map (f True (\v. not v)) xs
```
then GHC may well inline `f` at its call site, and thereby specialise it with `x=True` and ...See also
* #23293
* #22886
* #23931
* #23584 (refactor INLINE pragmas)
If you have
```
f x y z = <expr>
h xs = map (f True (\v. not v)) xs
```
then GHC may well inline `f` at its call site, and thereby specialise it with `x=True` and `y=\v.not v`, which is great.
But if you say `{-# INLINE f #-}` we insist on only inlning `f` [when it is applied to as many argument as appear on the LHS of the function defintion](https://ghc.gitlab.haskell.org/ghc/doc/users_guide/exts/pragmas.html?highlight=inline%20pragma#inline-pragma) (3 in this case). So adding INLINE will make the code optimise worse. That's not a good look. The regressions in #22886 are an example of what can happen.
So here's the proposal: **allow INLINE functions to inline even if not applied to the number of args on the LHS**.
Why do we have this "inline only if saturated" rule? The reasons seem to be lost in the mists of time. Clearly we should be rather cautious about changing this, but
* we have no documented reason for why it is a good idea
* it can make INLINE functions behave worse than non-INLINE ones
To mitigate the risks, one could imagine having a flag to let you get back the old behaiour.https://gitlab.haskell.org/ghc/ghc/-/issues/22902Optimization opportunity: hidden identity functions.2023-03-01T16:47:38ZJaro ReindersOptimization opportunity: hidden identity functions.<!--
READ THIS FIRST: If the feature you are proposing changes the language that GHC accepts
or adds any warnings to `-Wall`, it should be written as a [GHC Proposal](https://github.com/ghc-proposals/ghc-proposals/).
Other features, appr...<!--
READ THIS FIRST: If the feature you are proposing changes the language that GHC accepts
or adds any warnings to `-Wall`, it should be written as a [GHC Proposal](https://github.com/ghc-proposals/ghc-proposals/).
Other features, appropriate for a GitLab feature request, include GHC API/plugin
innovations, new low-impact compiler flags, or other similar additions to GHC.
-->
## Motivation
See [my discourse thread for the background story](https://discourse.haskell.org/t/a-solved-benchmarking-mystery/5738?u=jaror).
I am investigating if stream fusion could replace foldr/build fusion in base. During the investigation I've encountered bad benchmark results for the streamed `drop` function: it is up to **20x slower** than `Prelude.drop`. I've looked at the core and found that the streamed `drop` boiles down to a function like this:
```haskell
drop :: Int -> [a] -> [a]
drop 0 [] = []
drop 0 (x:xs) = x : drop 0 xs -- [1]
drop n [] = []
drop n (x:xs) = drop (n - 1) xs
```
The main difference with `Prelude.drop` is that this function walks over the remainder of the list that it does not drop (see the recursive call at [1]). That is extremely inefficient. In my benchmarks it is up to 20x slower than `Prelude.drop` because of the extra allocation and subsequent copying of the remainder of the list after dropping.
## Proposal
I think this function could be optimized further. First of all, SpecConstr could split it into two:
```haskell
drop :: Int -> [a] -> [a]
drop 0 [] = []
drop 0 (x:xs) = x : drop0 xs
drop n [] = []
drop n (x:xs) = drop (n - 1) xs
drop0 :: [a] -> [a]
drop0 [] = []
drop0 (x:xs) = x : drop0 xs
```
SpecConstr doesn't do this optimization yet, but #22781 tracks that issue.
Then a special optimization pass could recognize that `drop0` is an identity function and optimize it further to:
```haskell
drop :: Int -> [a] -> [a]
drop 0 [] = []
drop 0 (x:xs) = x : drop0 xs
drop n [] = []
drop n (x:xs) = drop (n - 1) xs
drop0 :: [a] -> [a]
drop0 xs = xs
```
This optimization could simply look for the pattern:
```
<f> xs =
case xs of
[] -> []
x : xs -> x : <f> xs
```
In Core and replace that with `<f> xs = xs`.
It also shouldn't be hard to generalize that for any ADT.
Then finally the result of that optimization already simplifies to (modulo unboxing):
```haskell
drop :: Int -> [a] -> [a]
drop 0 xs = xs
drop n [] = []
drop n (x:xs) = drop (n - 1) xs
```
Which is what we want.https://gitlab.haskell.org/ghc/ghc/-/issues/22312WorkWrap/FloatOut: Float out strictly unboxed free variables2022-10-25T14:34:48ZSebastian GrafWorkWrap/FloatOut: Float out strictly unboxed free variablesConsider
```hs
f :: (Int,Int) -> Int -> Int
f _ 0 = 0
f x y = g y
where
g y = case x of
(p,q) -> if p+y > 0 then 0
else g (y-1)
```
Note that `f` is not strict in `x`, but `g` is and doesn't even n...Consider
```hs
f :: (Int,Int) -> Int -> Int
f _ 0 = 0
f x y = g y
where
g y = case x of
(p,q) -> if p+y > 0 then 0
else g (y-1)
```
Note that `f` is not strict in `x`, but `g` is and doesn't even need the box. With `-O` (in the absence of LiberateCase), we fail to float out and unbox `x`:
```
f = \ (ds_d1pt :: (Int, Int)) (ds1_d1pu :: Int) ->
case ds1_d1pu of { GHC.Types.I# ds2_d1px ->
case ds2_d1px of ds3_X2 {
__DEFAULT ->
joinrec {
$wg_s1re [InlPrag=NOUSERINLINE[2], Occ=LoopBreaker]
:: GHC.Prim.Int# -> Int
[LclId[JoinId(1)], Arity=1, Str=<L> {d1pt->S!P(S!P(L),A)}, Unf=OtherCon []]
$wg_s1re (ww_s1rc :: GHC.Prim.Int#)
= case ds_d1pt of { (p_atY, q_atZ) ->
case p_atY of { GHC.Types.I# x_a1q0 ->
case GHC.Prim.># (GHC.Prim.+# x_a1q0 ww_s1rc) 0# of {
__DEFAULT -> jump $wg_s1re (GHC.Prim.-# ww_s1rc 1#);
1# -> Lib.f1
}
}
}; } in
jump $wg_s1re ds3_X2;
0# -> Lib.f1
}
}
```
Hence we keep on switching over it in `$wg` instead of just closing over the unboxed `x_a1q0`.
We float lazy work *inside* lambdas called at most once; hence we should also float strict work *outside* lambdas known to be called strictly and possibly many times.
I'm not sure what the best place to do this would be and it seems we lack the information of the outermost possible context in which `x` is used strictly.
But given we had that information, or at least knew "`x` is used strictly in the whole `joinrec`", then
1. We could do the transformation in WW, wrapping an unboxing case around the `joinrec` and rebox it in `$wg`'s body, or
2. We could tweak FloatOut so that it floats out `case x of (p,q) ->` if `x` is strict and unboxed.
Inspired by https://gitlab.haskell.org/ghc/ghc/-/issues/22303. In traditional Dataflow analysis terms, strictness analysis is like very busy expression analysis and FloatOut is like the code hoisting transformation https://ethz.ch/content/dam/ethz/special-interest/infk/inst-cs/lst-dam/documents/Education/Classes/Fall2015/210_Compiler_Design/Slides/extra.pdf.https://gitlab.haskell.org/ghc/ghc/-/issues/22179DmdAnal/Meta: Is strictness the right property to look for?2023-12-20T04:48:20ZSebastian GrafDmdAnal/Meta: Is strictness the right property to look for?In #19917, we have seen how strictness analysis can be "too aggressive". Small, executable reproducer:
```hs
main = print $ loop "foo" (error "woops")
where
loop xs l = case xs of
x:xs -> loop xs x :: String
[] -> l ...In #19917, we have seen how strictness analysis can be "too aggressive". Small, executable reproducer:
```hs
main = print $ loop "foo" (error "woops")
where
loop xs l = case xs of
x:xs -> loop xs x :: String
[] -> l `seq` loop [l] l
```
This program will crash with `"woops"` when compiled with optimisations, because `loop` is inferred to be strict in its `l` parameter. That is *very* surprising! Note that `l` doesn't even *occur freely* in the cons branch of `loop`. And neither will `error "woops"` be evaluated under call-by-need; instead we'll get an infinite loop.
On the other hand, `loop`'s parameter `l` certainly *does* satisfy the strictness definition: Whenever `l` is bottom/diverges, `loop xs l` will, too, simply because `loop` *always* diverges.
That is to say: The above program points out that The Strictness Definition of Olde entirely neglects diverging program traces, simply because Strictness is a property of denotational semantics and all such program traces share the same denotation ⊥.
I think a more useful definition is
> *Strong Strictness*. `f` is strong strict if `f x` will evaluate `x` after finitely many steps in the small-step semantics.
NB: For terminating programs/traces, strong strictness and strictness coincide. It is only about diverging traces where we see the difference.
`loop` above is strong strict in the first but not in its second argument, as `loop "foo" (error "woops")` will loop forever without ever throwing that `error "woops"`. It is still strong strict in `xs`. Nice!
Unfortunately, I'm pretty sure that this stronger notion of strictness would regress performance all over the place. For example, `\x -> error "blah"` would not be strong strict (in `x`, obviously). But I think it's good to describe the ideal model so that one day we might even implement it.
---
Here is a bit Related Work:
1. Usage analysis as implemented in GHC actually works for diverging programs, too: We'd never say that `loop` above *does not* use its first argument, although `loop` diverges. Usage/Absence analysis has to account for diverging/infinite traces to be useful.
2. What Strictness Analysis is to *variables* in lazy functional languages is quite similar to what [Very Busy Expression Analysis](https://web.cs.wpi.edu/~kal/PLT/PLT9.6.html) is to *expressions* in an imperative setting. Very Busy Expression Analysis is a backwards analysis with a MUST powerset lattice (e.g., ⊥ is *all* Exprs, lub is intersection). I investigated how this analysis handles infinite loops. It turns out that infinite loops are very busy *in every expression*, even if it doesn't occur in the loop. Pretty much the same problem as for us!
3. Then I recalled that the gen/kill formulation for Post-dominance analysis is *also* a backwards analysis with a MUST powerset lattice and actually shares the all the same problems wrt. infinite loops. And sure enough, there is an abundance of discussion and literature about post-dominance and infinite loops:
- https://stackoverflow.com/questions/35399281/how-can-i-build-the-post-dominator-tree-of-a-function-with-an-endless-loop discusses how LLVM was failing to properly handle infinite loops in their post dominance algorithm. I *think* it has been fixed, though: https://reviews.llvm.org/D35851. Also it's a bit of a different problem they had: Their solutions were too weak (e.g., Top), ours are too optimistic (e.g., report too many vars that we are strict in).
- [A Formal Model of Program Dependences and its Implications for Software Testing, Debugging, and Maintenance](https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=58784) introduces the notion of *strong* post-dominance (they called it strong forward dominance), by
> u strongly post dominates v <=> u post dominates v and there is an integer k > 0 such that every walk in G beginning with v and of length >= k contains u.
And that's indeed why I called my definition above Strong Strictness: It has a an additional "in finitely many steps" condition compared to baseline Strictness.
---
Implementation-wise, I think we can do the following during fixed-point iteration (I don't want to touch strictness of `error` just yet):
1. We keep initialising strictness signatures of recursive functions such as `loop` with bottom (meaning an application of `loop` is strict in every var).
2. We do one round of fixed-point iteration and get to find a too optimistic signature that involves *just the arguments and free variables that occur during any finite evaluation of `loop`*. E.g., we'll get something like `<1B><1B>b`.
3. Now we'll do another iteration, but we'll omit the `b`, replacing it with `x` (`exnDiv`, could have chosen `topDiv` too). NB: We'll keep the argument demands and demands on free vars we accumulated, because they actually occur in finitely many steps.
4. Then onto the next iteration: From the `x:xs` branch, we'll see that we are strict in `x` and `xs`, but not in `l` (because we switched from `b` to `x`!). So when we lub together with the `[]` branch, we see `{l:->L}`.
And then we'll get `<1B><L>x`, which is perfect! Still strict in the first arg, no longer strict in the second and we get to see through `x` that evaluation of `loop` will never terminate, hence dead-code elimination will keep working as before.
What happened in (3) is that we basically used `b` to figure out in the first iteration *just the right set* of FVs that `loop` has to be strict in in the recursive call. We could've instead done a special FV traversal that accounts for FVs in strictness signatures and initialised the set of occuring FVs to demand `B` instead, I suppose. Maybe we should do that instead... The important part is that instead of saying that `loop` is strict in *every* variable during the first iteration, we say that it's strict in *just the sensible set of FVs* instead.
I put up an experimental implementation of this idea in !8983.https://gitlab.haskell.org/ghc/ghc/-/issues/21872Modularity for the Core optimization passes and linter2022-10-17T22:50:02ZJohn EricsonModularity for the Core optimization passes and linter_This part of #17957, split off to its own issue to more thoroughly elaborate the design rather than just the general principles._
--------
## Code we have to organize
Right now, here is what there is:
1. A few of Core -> Core optimi..._This part of #17957, split off to its own issue to more thoroughly elaborate the design rather than just the general principles._
--------
## Code we have to organize
Right now, here is what there is:
1. A few of Core -> Core optimizations passes
2. Logic to lint Core
3. "end pass" logic to do misc dumping and linting used by both the core -> core passes *and* other pipeline stages where the output is Core but the input might be something else.
4. `CoreToDo` which is "mostly" an enum for each Core -> Core pass, but also the other uses of the end pass logic shoe-horned in.
5. `GHC.Core.Opt.Pipeline`, which contains a mix of driver and non-driver code
- ```haskell
getCoreToDo :: Logger -> DynFlags -> [CoreToDo]
```
which plans which Core -> Core passes should be run based on `DynFlags`
- ```haskell
runCorePasses :: [CoreToDo] -> ModGuts -> CoreM ModGuts
```
which executes the plan, making a modified `ModGuts`
## Issues with the code before we started
### `DynFlags` and `HscEnv` used all over the place
This is a classic issue very much not distinctive to this part of the compiler, discussed at length in the paper.
### Domain-layer linting logic is mixed with application-layer / presentation-layer "end pass logic"
The former is an essential part of GHC, *defining* what well-formed Core looks like. The latter is just a mere convention downstream of the definition of Core well-formedness
`GHC.Core.Opt.Pipeline` is also a mix of application and above layer and domain logic because because it contains small bits of domain logic alone with driver logic.
### `CoreToDo` begets partiality
The variants that *don't* correspond to `CoreToCore` passes cause panics if they are encountered in `runCorePasses`. We are relying on `getCoreToDo` never outputting them, and crossing fingers.
This is done because a `CoreToDo` is required to use the end pass logic, and so the other non Core -> Core pass uses of the end pass logic need their own variants.
### Individual passes use `CoreToCore`
The is a general software design principle that "things should not know their own names".
The purpose of `CoreToCore` is to enumerate *all* of the Core -> Core passes, but this should be downstream of each pass so the pass *by construction* is only aware of itself and not how its used.
However, since the end-pass logic requires passing a `CoreToCore`, each pass must import the `CoreToCore` datatype in order to pass its own variant to the end pass logic.
This creates and anti-modular confusion where `CoreToCore`, which represents the downstream code using all the Core -> Core passes, is actually defined upstream of each pass.
## New design addressing issues
### Each Core -> Core pass should stand alone
Every Core -> Core pass should be considered its own component, and thus receive the general treatment described in the paper.
To recap:
- Rather than taking `DynFlags`, it should take a configuration record with just the items this component cares about.
- Module `GHC.Foo` has a companion module `GHC.Driver.Config.Foo` which translate `DynFlags` to the per-component configuration record purely.
- Rather than taking `HscEnv`, it should just take the "services" that it cares about.
- The paper recommended that each service be passed separately but where there are concerns of about state getting out of sync, we are happy to compromise and make a smaller version of `HscEnv` bundling the services together.
- We could make a `GHC.Driver.Env.Foo` module for the `HscEnv` to mini-`HscEnv` translation.
Note that we shouldn't confused *external configuration* with *internal state*.
In https://gitlab.haskell.org/ghc/ghc/-/merge_requests/8598/diffs#note_441105, @simonpj asks why a new configuration record is being added. The answer is that the existing `SimplTopEnv` is used in the definition of a monad *internal* to the simplifier. It would be a *regression* in modularity to force the caller of the simplifier to be aware of this monad when it was previously a mere implementation detail.
By creating a new record with just the parts of the monad env that are set by the outside world, we preserve the encapsulation boundary allowing the outside world to continue to not need to know/care about information it isn't responsible for providing.
**SPJ**: OK but let's articulate that logic explicitly with the data types. There are rather a lot of them now.
### The end-pass logic is split out from `GHC.Core.Lint` in to `GHC.Core.EndPass`
Domain-layer- and application-layer- logic is thus no longer mixed together.
Both are treated as components and get per-component configuration records accordingly.
### Neither the end-pass logic nor linting logic should take any `CoreToDo`s. And knock on effects.
`CoreToDo` is expunged from their nascent configure records. This enables these additional problem-resolving changes:
1. The individual passes which now provide end pass configs *also* don't need to use `CoreToDo` either, and can instead directly pass the metadata which was dispatched from the `CoreToDo` directly.
2. Passes that are not Core -> Core passes can be purged from `CoreToDo`. Those passes too skip using `CoreToDo` to provide information to `EndPass`. Those removed usages were the sole use-case of those improper `CoreToDo` variants, so now we remove them. This removes the partiality in `GHC.Driver.Pipeline`.
2. `CoreToDo` need no longer be imported by any individual pass, but just be the logic putting everything together.
### Confine `CoreM` to the plugin use-case.
`CoreM` currently takes/provides a lot of information. This is overkill for the regular built-in Core-> Core passes that use it, and antithetical to our genral approach of modularizing components.
However, plugins are a different, special case. We do no know what plugins do ahead of time, and so to be generous we wish to provide them with as much information as possible. That means we *should* continue to give plugins the "kitchen sink" `CoreM`.
That means we get rid of `CoreM` everywhere else besides the plugins. It should be use by the plugins and the driver code which calls the plugins only.
### Plan more completely
`CoreToDo` variants should also contain the new configuration records of the pass they refer to. Then `getCoreToDo` can more completely plan out what needs to be done by elaborating `DynFlags` into both the list of passes to be run (in order) *and* the configuration for each pass. This is the complete Core -> Core plan.
### Split `GHC.Core.Opt.Pipeline` and rename
The non-driver bits of domain logic should be given their own homes in other module(s). The remain driver logic (including the principle functions we've mentioned in this issue) will remain, and so the module should renamed `GHC.Driver.Core.Opt`.
This will uphold our convention (to eventually be enforced by lint), that `DynFlags` and `HscEnv` should only appear in `GHC.Driver` modules.
## Tasks
- [ ] Write a note "Configuration records in GHC". Related discussion: https://gitlab.haskell.org/ghc/ghc/-/merge_requests/8598#note_443781
- [ ] Write a note "The GHC.Driver.Config namespace". Related discussion: https://gitlab.haskell.org/ghc/ghc/-/merge_requests/8598#note_443963
## References
The parent issue: #17957
The paper: https://hsyl20.fr/home/posts/2022-05-03-modularizing-ghc-paper.html
The wiki page: https://gitlab.haskell.org/ghc/ghc/-/wikis/Make-GHC-codebase-more-modularhttps://gitlab.haskell.org/ghc/ghc/-/issues/21611Use `CoreM` just for core plugins; `CoreM` stays maximal2022-09-06T20:44:50ZDominik PetelerUse `CoreM` just for core plugins; `CoreM` stays maximal## Problem
To solve #17957, we refactor GHC again and again, component by component to follow the so-called ["Law of Demeter"](https://en.wikipedia.org/wiki/Law_of_Demeter).
The problem is, while regular GHC components do a clear opera...## Problem
To solve #17957, we refactor GHC again and again, component by component to follow the so-called ["Law of Demeter"](https://en.wikipedia.org/wiki/Law_of_Demeter).
The problem is, while regular GHC components do a clear operation with clear requirements, plugins are black boxes supplied by the user of GHC. We don't know what they are doing (and couldn't, because yet-to-be-written plugins form an open world), and thus we cannot pass them just what is needed because that vary notion is ill-defined.
If anything, we should do the opposite for plugins: not knowing what they need we could error on passing them as *much* state as possible, giving them utmost flexibility.¹
## Solution
Given these issues, the prudent option seems to be to not break existing plugins, passing them the same kitchen sink of state as before. Plugins use `CoreM` today, and so they continue to use `CoreM` tomorrow, and `CoreM` itself remains unchanged.
What about #17957? If instead of envisioning the entire Core optimizer as a single component, we instead look at each *pass* as a component, we get ourselves unstuck.
- The passes that are part of GHC are regular code rather than opaque plugin parameters, and so we can do our normal "Law of Demeter" work. They would not longer use `CoreM`.
- `GHC.Core.Opt.Pipeline` contains some core pass logic that should probably be moved to another module, but the main "coordinating/planning" code part of it, which would continue to dispatch to core plugins too, can move under `GHC.Driver`, where manipulating the "whole state" is not verboten.
- `CoreM` should therefore live in `GHC.Plugins` and not in the Core optimizer namespace.
This issue tracks the first untangling step, which is making `CoreM` just be used by the core plugins. The remaining work can then proceed per pass in parallel.
## Example
The specialization pass in `GHC.Core.Opt.Specialise` currently uses two environments:
- `SpecEnv`:
```haskell
data SpecEnv
= SE { se_dflags :: DynFlags
, ...
}
``
- the one stored in `CoreM`:
```haskell
data CoreReader = CoreReader {
cr_hsc_env :: HscEnv,
...
}
```
After the refactoring the pass will run in the `IO` monad and the parts of the `CoreReader` type relevant to this pass will be contained in `SpecEnv`.
Because of this, after the refactoring the only remaining users of this monad are Core2core plugins.
-------
1: Full disclosure, in the *way long term*, @ericson2314 thinks this is why plugins at all are not the right solution and a modular GHC can offer another way, but we're not trying to push that agenda here nor any time soon.Dominik PetelerDominik Petelerhttps://gitlab.haskell.org/ghc/ghc/-/issues/21562Make use of boxity analysis in SpecConstr.2024-02-27T13:58:43ZAndreas KlebingerMake use of boxity analysis in SpecConstr.It seems we put a lot of work into figuring out already if unboxing an argument is safe to do or not.
We should make use of that for SpecConstr which currently will always take apart boxed arguments if it can.
The main difference for S...It seems we put a lot of work into figuring out already if unboxing an argument is safe to do or not.
We should make use of that for SpecConstr which currently will always take apart boxed arguments if it can.
The main difference for SpecConstr is that we *also* want to apply this logic to lazy arguments. I'm not sure how hard that would be to achieve.
But even just not taking apart strict arguments if boxity indicates we shouldn't would be an improvement.https://gitlab.haskell.org/ghc/ghc/-/issues/21540Deprecation of GHC.Pack (phase 2)2023-09-07T09:36:13ZHécate MoonlightDeprecation of GHC.Pack (phase 2)In https://gitlab.haskell.org/ghc/ghc/-/issues/21461 the decision has been taken to deprecate GHC.Pack.
This ticket tracks the phase 2 of the three-releases deprecation cycle, where a warning is thrown when people import the module.In https://gitlab.haskell.org/ghc/ghc/-/issues/21461 the decision has been taken to deprecate GHC.Pack.
This ticket tracks the phase 2 of the three-releases deprecation cycle, where a warning is thrown when people import the module.9.10.1https://gitlab.haskell.org/ghc/ghc/-/issues/21431Namespace CPP macros2022-04-29T08:54:28ZBen GamariNamespace CPP macrosCurrently GHC exposes quite a number of CPP macros with generic names (e.g. `PROFILING`, `THREADED_RTS`) to user code. This can cause trouble when integrating GHC into larger systems. Arguably we should be more careful about prefixing ma...Currently GHC exposes quite a number of CPP macros with generic names (e.g. `PROFILING`, `THREADED_RTS`) to user code. This can cause trouble when integrating GHC into larger systems. Arguably we should be more careful about prefixing macro names (e.g. with `GHC_`) to ensure that we do not pollute the CPP namespace.