GitLab

This commits relaxes the invariants of the Core syntax so that a
top-level variable can be bound to a primitive string literal of type
Addr#.

This commit:

* Relaxes the invatiants of the Core, and allows top-level bindings whose
  type is Addr# as long as their RHS is either a primitive string literal or
  another variable.

* Allows the simplifier and the full-laziness transformer to float out
  primitive string literals to the top leve.

* Introduces the new StgGenTopBinding type to accomodate top-level Addr#
  bindings.

* Introduces a new type of labels in the object code, with the suffix "_bytes",
  for exported top-level Addr# bindings.

* Makes some built-in rules more robust. This was necessary to keep them
  functional after the above changes.

This is a continuation of D2554.

Rebasing notes:
This had two slightly suspicious performance regressions:

* T12425: bytes allocated regressed by roughly 5%
* T4029: bytes allocated regressed by a bit over 1%
* T13035: bytes allocated regressed by a bit over 5%

These deserve additional investigation.

Rebased by: bgamari.

Test Plan: ./validate --slow

Reviewers: goldfire, trofi, simonmar, simonpj, austin, hvr, bgamari

Reviewed By: trofi, simonpj, bgamari

Subscribers: trofi, simonpj, gridaphobe, thomie

Differential Revision: https://phabricator.haskell.org/D2605

GHC Trac Issues: #8472

Trac #13156 showed a lost opportunity for CSE. I found that it was
easy to fix, and it had the nice side effect of rendering a previous
nasty case, described in Note [Corner case for case expressions],
unnecessary.

Simpler code, does more.  Great.

I noticed that CSE.addBinding was always returning one of its own
inputs, so I refactored to avoid doing so.

No change in behaviour.

It turned out that many different modules defined the same type
synonyms (InId, OutId, InType, OutType, etc) for the same purpose.

This patch is refactoring only: it moves all those definitions to
CoreSyn.

I had the environments wrong so that CSE could mis-clone
an expression, if the uniques just happened to be badly
arranged.  It's hard to trigger the bug, so I can't make
a reliable test case.

Happily the fix is easy.

Triggered by an observation by Joachim, Simon felt the urge to clean up
the CSE code a bit. This is the result.

(Code by Simon, commit message and other leg-work by Joachim)

Differential Revision: https://phabricator.haskell.org/D2074

* Comments to explain that a CoVar, whose IdInfo is CoVarId,
  is always unlifted (but may be nominal or representational role)

  And TyCoRep.isCoercionType picks out only those unlifted
  types, NOT the lifted versions

* Introduce Var.NcId for non-co-var Ids
  with predicate isNonCoVarId

* Add assertions in CoreSubst that the Id env is only
  used for NcIds

* Fix lurking bug in CSE which extended the
  CoreSubst Id env with a CoVar

* Fix two bugs in Specialise.spec_call, which wrongly treated
  CoVars like NcIds
    - needed a varToCoreExpr in one place
    - needed extendSubst not extendIdSubst in another
  This was the root cause of Trac #11644

Minor refactoring

* Eliminate unused mkDerivedLocalCoVarM, mkUserLocalCoVar
* Small refactor in mkSysLocalOrCoVar

This implements the ideas originally put forward in
"System FC with Explicit Kind Equality" (ICFP'13).

There are several noteworthy changes with this patch:
 * We now have casts in types. These change the kind
   of a type. See new constructor `CastTy`.

 * All types and all constructors can be promoted.
   This includes GADT constructors. GADT pattern matches
   take place in type family equations. In Core,
   types can now be applied to coercions via the
   `CoercionTy` constructor.

 * Coercions can now be heterogeneous, relating types
   of different kinds. A coercion proving `t1 :: k1 ~ t2 :: k2`
   proves both that `t1` and `t2` are the same and also that
   `k1` and `k2` are the same.

 * The `Coercion` type has been significantly enhanced.
   The documentation in `docs/core-spec/core-spec.pdf` reflects
   the new reality.

 * The type of `*` is now `*`. No more `BOX`.

 * Users can write explicit kind variables in their code,
   anywhere they can write type variables. For backward compatibility,
   automatic inference of kind-variable binding is still permitted.

 * The new extension `TypeInType` turns on the new user-facing
   features.

 * Type families and synonyms are now promoted to kinds. This causes
   trouble with parsing `*`, leading to the somewhat awkward new
   `HsAppsTy` constructor for `HsType`. This is dispatched with in
   the renamer, where the kind `*` can be told apart from a
   type-level multiplication operator. Without `-XTypeInType` the
   old behavior persists. With `-XTypeInType`, you need to import
   `Data.Kind` to get `*`, also known as `Type`.

 * The kind-checking algorithms in TcHsType have been significantly
   rewritten to allow for enhanced kinds.

 * The new features are still quite experimental and may be in flux.

 * TODO: Several open tickets: #11195, #11196, #11197, #11198, #11203.

 * TODO: Update user manual.

Tickets addressed: #9017, #9173, #7961, #10524, #8566, #11142.
Updates Haddock submodule.

Trac #10218 reports a subtle bug that turned out to be:

- CSE invalidated the usage information computed
  by earlier demand analysis, by increasing sharing

- that made a single-entry thunk into a multi-entry thunk

- and with -feager-blackholing, that led to <<loop>>

The patch fixes it by making the CSE pass zap usage information for
let-bound identifiers.   It can be restored by -flate-dmd-anal.

(But making -flate-dmd-anal the default needs some careful work;
see Trac #7782.)

As with stripTicksTop, this is because we often need the stripped
expression but not the ticks (at least not right away). This makes a big
difference for CSE, see #9961.
Signed-off-by: Austin Seipp <austin@well-typed.com>

This patch introduces "SourceNote" tickishs that link Core to the
source code that generated it. The idea is to retain these source code
links throughout code transformations so we can eventually relate
object code all the way back to the original source (which we can,
say, encode as DWARF information to allow debugging).  We generate
these SourceNotes like other tickshs in the desugaring phase. The
activating command line flag is "-g", consistent with the flag other
compilers use to decide DWARF generation.

Keeping ticks from getting into the way of Core transformations is
tricky, but doable. The changes in this patch produce identical Core
in all cases I tested -- which at this point is GHC, all libraries and
nofib. Also note that this pass creates *lots* of tick nodes, which we
reduce somewhat by removing duplicated and overlapping source
ticks. This will still cause significant Tick "clumps" - a possible
future optimization could be to make Tick carry a list of Tickishs
instead of one at a time.

(From Phabricator D169)

Signed-off-by: Austin Seipp <austin@well-typed.com>

Thanks to Gabor for pointing this out

Fixes Trac #9529

Previusly we simply weren't doing CSE at all on things
whose unfolding were not always-active, for reasons explained
in Note [CSE for INLINE and NOINLINE].  But that was bad!
Making something INLNEABLE meant that its RHS was no longer
CSE'd, and that made some nofib programs worse.

And it's entirely unnecessary.  I thoguht it through again,
wrote new comments (under the same Note), and things are
better again.

In some cases, the layout of the LANGUAGE/OPTIONS_GHC lines has been
reorganized, while following the convention, to

- place `{-# LANGUAGE #-}` pragmas at the top of the source file, before
  any `{-# OPTIONS_GHC #-}`-lines.

- Moreover, if the list of language extensions fit into a single
  `{-# LANGUAGE ... -#}`-line (shorter than 80 characters), keep it on one
  line. Otherwise split into `{-# LANGUAGE ... -#}`-lines for each
  individual language extension. In both cases, try to keep the
  enumeration alphabetically ordered.
  (The latter layout is preferable as it's more diff-friendly)

While at it, this also replaces obsolete `{-# OPTIONS ... #-}` pragma
occurences by `{-# OPTIONS_GHC ... #-}` pragmas.

This patch fixes a bad omission in CSE, thanks to 'michaelt' for spotting
it, and correctly identifying the fix (in cseRhs).  The trouble was with
   x1 = C a b
   x2 = C x1 b
   y1 = C a b
   y2 = C y1 b
we were not commoning up y2=x2, because we failed to substitute y1:=x1,
so y2's RHS looked different to x2's

I also refactoring, so taht the cs_map in a CSEnv map is
       cs_map    :: CoreMap (OutExpr, Id)
instead of
       cs_map    :: CoreMap (OutExpr, OutExpr)
Much nicer!

This doesn't make much difference to allocation, but it gives a surprisingly
big benefit to binary size.

--------------------------------------------------------------------------------
        Program           Size    Allocs   Runtime   Elapsed  TotalMem
--------------------------------------------------------------------------------
           ansi          -1.7%     -0.8%      0.00      0.00     +0.0%
           bspt          -1.6%     -1.5%      0.01      0.01     +0.0%
      cacheprof          -1.8%     -0.2%     +1.6%     +1.9%     +2.7%
            fft          -1.4%     -1.3%      0.06      0.06    +11.1%
            ida          -1.4%     -1.0%      0.12      0.12     +0.0%
           rfib          -1.4%     -0.1%      0.03      0.03     +0.0%
            scs          -1.6%     -0.1%     +1.5%     +1.5%     +0.0%
  spectral-norm          -1.3%     -0.1%     -0.2%     -0.2%     +0.0%
            tak          -1.4%     -0.1%      0.02      0.02     +0.0%
        veritas          -1.4%     -0.1%      0.00      0.00     +0.0%
--------------------------------------------------------------------------------
            Min          -2.5%     -1.5%    -11.8%    -11.8%     -8.0%
            Max          -1.0%     +0.0%     +2.7%     +2.5%    +11.1%
 Geometric Mean          -1.3%     -0.1%     -2.6%     -2.6%     +0.0%

See Note [CSE for case expressions].  I don't think this is a big
deal, but it's nice, and it's easy.

This is done by a 'unarisation' pre-pass at the STG level which
translates away all (live) binders binding something of unboxed
tuple type.

This has the following knock-on effects:
  * The subkind hierarchy is vastly simplified (no UbxTupleKind or ArgKind)
  * Various relaxed type checks in typechecker, 'foreign import prim' etc
  * All case binders may be live at the Core level

We only use it for "compiler" sources, i.e. not for libraries.
Many modules have a -fno-warn-tabs kludge for now.

User visible changes
====================

Profilng
--------

Flags renamed (the old ones are still accepted for now):

  OLD            NEW
  ---------      ------------
  -auto-all      -fprof-auto
  -auto          -fprof-exported
  -caf-all       -fprof-cafs

New flags:

  -fprof-auto              Annotates all bindings (not just top-level
                           ones) with SCCs

  -fprof-top               Annotates just top-level bindings with SCCs

  -fprof-exported          Annotates just exported bindings with SCCs

  -fprof-no-count-entries  Do not maintain entry counts when profiling
                           (can make profiled code go faster; useful with
                           heap profiling where entry counts are not used)

Cost-centre stacks have a new semantics, which should in most cases
result in more useful and intuitive profiles.  If you find this not to
be the case, please let me know.  This is the area where I have been
experimenting most, and the current solution is probably not the
final version, however it does address all the outstanding bugs and
seems to be better than GHC 7.2.

Stack traces
------------

+RTS -xc now gives more information.  If the exception originates from
a CAF (as is common, because GHC tends to lift exceptions out to the
top-level), then the RTS walks up the stack and reports the stack in
the enclosing update frame(s).

Result: +RTS -xc is much more useful now - but you still have to
compile for profiling to get it.  I've played around a little with
adding 'head []' to GHC itself, and +RTS -xc does pinpoint the problem
quite accurately.

I plan to add more facilities for stack tracing (e.g. in GHCi) in the
future.

Coverage (HPC)
--------------

 * derived instances are now coloured yellow if they weren't used
 * likewise record field names
 * entry counts are more accurate (hpc --fun-entry-count)
 * tab width is now correct (markup was previously off in source with
   tabs)

Internal changes
================

In Core, the Note constructor has been replaced by

        Tick (Tickish b) (Expr b)

which is used to represent all the kinds of source annotation we
support: profiling SCCs, HPC ticks, and GHCi breakpoints.

Depending on the properties of the Tickish, different transformations
apply to Tick.  See CoreUtils.mkTick for details.

Tickets
=======

This commit closes the following tickets, test cases to follow:

  - Close #2552: not a bug, but the behaviour is now more intuitive
    (test is T2552)

  - Close #680 (test is T680)

  - Close #1531 (test is result001)

  - Close #949 (test is T949)

  - Close #2466: test case has bitrotted (doesn't compile against current
    version of vector-space package)

and comment its invariants in Note [CoreProgram] in CoreSyn

I'm not totally convinced that CoreProgram is the right name
(perhaps CoreTopBinds might better), but it is useful to have
a clue that you are looking at the top-level bindings.

This is only a matter of a type synonym change; no deep
refactoring here.

For CSE it's obviously great to have a mapping whose
key is an expression.  This patch makes CSE use the
new CoreTrie data type.

I did some very simple performance comparisions. The
change in compile-time allocation is less than 1%,
but it does go down!  Slightly.

It was inconsistent before, now it's right

See the paper "Practical aspects of evidence based compilation in System FC"

* Coercion becomes a data type, distinct from Type

* Coercions become value-level things, rather than type-level things,
  (although the value is zero bits wide, like the State token)
  A consequence is that a coerion abstraction increases the arity by 1
  (just like a dictionary abstraction)

* There is a new constructor in CoreExpr, namely Coercion, to inject
  coercions into terms

In CSE we were getting lots of apprarently-unequal expressions with
the same hash code.  In fact they were perfectly equal -- but we were
using a cheap-and-cheerful equality tests for CoreExpr that said False
for any lambda expression!

This patch adds a proper equality test for Core, with alpha-renaming.
It's easy to do, and will avoid silly cases of CSE failing to fire.

We should get less of this:
  WARNING: file compiler/simplCore/CSE.lhs line 326
  extendCSEnv: long list, length 18
from a compiler built with -DDEBUG

This patch has been a long time in gestation and has, as a
result, accumulated some extra bits and bobs that are only
loosely related.  I separated the bits that are easy to split
off, but the rest comes as one big patch, I'm afraid.

Note that:
 * It comes together with a patch to the 'base' library
 * Interface file formats change slightly, so you need to
   recompile all libraries

The patch is mainly giant tidy-up, driven in part by the
particular stresses of the Data Parallel Haskell project. I don't
expect a big performance win for random programs.  Still, here are the
nofib results, relative to the state of affairs without the patch

        Program           Size    Allocs   Runtime   Elapsed
--------------------------------------------------------------------------------
            Min         -12.7%    -14.5%    -17.5%    -17.8%
            Max          +4.7%    +10.9%     +9.1%     +8.4%
 Geometric Mean          +0.9%     -0.1%     -5.6%     -7.3%

The +10.9% allocation outlier is rewrite, which happens to have a
very delicate optimisation opportunity involving an interaction
of CSE and inlining (see nofib/Simon-nofib-notes). The fact that
the 'before' case found the optimisation is somewhat accidental.
Runtimes seem to go down, but I never kno wwhether to really trust
this number.  Binary sizes wobble a bit, but nothing drastic.


The Main Ideas are as follows.

InlineRules
~~~~~~~~~~~
When you say 
      {-# INLINE f #-}
      f x = <rhs>
you intend that calls (f e) are replaced by <rhs>[e/x] So we
should capture (\x.<rhs>) in the Unfolding of 'f', and never meddle
with it.  Meanwhile, we can optimise <rhs> to our heart's content,
leaving the original unfolding intact in Unfolding of 'f'.

So the representation of an Unfolding has changed quite a bit
(see CoreSyn).  An INLINE pragma gives rise to an InlineRule 
unfolding.  

Moreover, it's only used when 'f' is applied to the
specified number of arguments; that is, the number of argument on 
the LHS of the '=' sign in the original source definition. 
For example, (.) is now defined in the libraries like this
   {-# INLINE (.) #-}
   (.) f g = \x -> f (g x)
so that it'll inline when applied to two arguments. If 'x' appeared
on the left, thus
   (.) f g x = f (g x)
it'd only inline when applied to three arguments.  This slightly-experimental
change was requested by Roman, but it seems to make sense.

Other associated changes

* Moving the deck chairs in DsBinds, which processes the INLINE pragmas

* In the old system an INLINE pragma made the RHS look like
   (Note InlineMe <rhs>)
  The Note switched off optimisation in <rhs>.  But it was quite
  fragile in corner cases. The new system is more robust, I believe.
  In any case, the InlineMe note has disappeared 

* The workerInfo of an Id has also been combined into its Unfolding,
  so it's no longer a separate field of the IdInfo.

* Many changes in CoreUnfold, esp in callSiteInline, which is the critical
  function that decides which function to inline.  Lots of comments added!

* exprIsConApp_maybe has moved to CoreUnfold, since it's so strongly
  associated with "does this expression unfold to a constructor application".
  It can now do some limited beta reduction too, which Roman found 
  was an important.

Instance declarations
~~~~~~~~~~~~~~~~~~~~~
It's always been tricky to get the dfuns generated from instance
declarations to work out well.  This is particularly important in 
the Data Parallel Haskell project, and I'm now on my fourth attempt,
more or less.

There is a detailed description in TcInstDcls, particularly in
Note [How instance declarations are translated].   Roughly speaking
we now generate a top-level helper function for every method definition
in an instance declaration, so that the dfun takes a particularly
stylised form:
  dfun a d1 d2 = MkD (op1 a d1 d2) (op2 a d1 d2) ...etc...

In fact, it's *so* stylised that we never need to unfold a dfun.
Instead ClassOps have a special rewrite rule that allows us to
short-cut dictionary selection.  Suppose dfun :: Ord a -> Ord [a]
                                            d :: Ord a
Then   
    compare (dfun a d)  -->   compare_list a d 
in one rewrite, without first inlining the 'compare' selector
and the body of the dfun.

To support this
a) ClassOps have a BuiltInRule (see MkId.dictSelRule)
b) DFuns have a special form of unfolding (CoreSyn.DFunUnfolding)
   which is exploited in CoreUnfold.exprIsConApp_maybe

Implmenting all this required a root-and-branch rework of TcInstDcls
and bits of TcClassDcl.


Default methods
~~~~~~~~~~~~~~~
If you give an INLINE pragma to a default method, it should be just
as if you'd written out that code in each instance declaration, including
the INLINE pragma.  I think that it now *is* so.  As a result, library
code can be simpler; less duplication.


The CONLIKE pragma
~~~~~~~~~~~~~~~~~~
In the DPH project, Roman found cases where he had

   p n k = let x = replicate n k
           in ...(f x)...(g x)....

   {-# RULE f (replicate x) = f_rep x #-}

Normally the RULE would not fire, because doing so involves 
(in effect) duplicating the redex (replicate n k).  A new
experimental modifier to the INLINE pragma, {-# INLINE CONLIKE
replicate #-}, allows you to tell GHC to be prepared to duplicate
a call of this function if it allows a RULE to fire.

See Note [CONLIKE pragma] in BasicTypes


Join points
~~~~~~~~~~~
See Note [Case binders and join points] in Simplify


Other refactoring
~~~~~~~~~~~~~~~~~
* I moved endPass from CoreLint to CoreMonad, with associated jigglings

* Better pretty-printing of Core

* The top-level RULES (ones that are not rules for locally-defined things)
  are now substituted on every simplifier iteration.  I'm not sure how
  we got away without doing this before.  This entails a bit more plumbing
  in SimplCore.

* The necessary stuff to serialise and deserialise the new
  info across interface files.

* Something about bottoming floats in SetLevels
      Note [Bottoming floats]

* substUnfolding has moved from SimplEnv to CoreSubs, where it belongs


--------------------------------------------------------------------------------
        Program           Size    Allocs   Runtime   Elapsed
--------------------------------------------------------------------------------
           anna          +2.4%     -0.5%      0.16      0.17
           ansi          +2.6%     -0.1%      0.00      0.00
           atom          -3.8%     -0.0%     -1.0%     -2.5%
         awards          +3.0%     +0.7%      0.00      0.00
         banner          +3.3%     -0.0%      0.00      0.00
     bernouilli          +2.7%     +0.0%     -4.6%     -6.9%
          boyer          +2.6%     +0.0%      0.06      0.07
         boyer2          +4.4%     +0.2%      0.01      0.01
           bspt          +3.2%     +9.6%      0.02      0.02
      cacheprof          +1.4%     -1.0%    -12.2%    -13.6%
       calendar          +2.7%     -1.7%      0.00      0.00
       cichelli          +3.7%     -0.0%      0.13      0.14
        circsim          +3.3%     +0.0%     -2.3%     -9.9%
       clausify          +2.7%     +0.0%      0.05      0.06
  comp_lab_zift          +2.6%     -0.3%     -7.2%     -7.9%
       compress          +3.3%     +0.0%     -8.5%     -9.6%
      compress2          +3.6%     +0.0%    -15.1%    -17.8%
    constraints          +2.7%     -0.6%    -10.0%    -10.7%
   cryptarithm1          +4.5%     +0.0%     -4.7%     -5.7%
   cryptarithm2          +4.3%    -14.5%      0.02      0.02
            cse          +4.4%     -0.0%      0.00      0.00
          eliza          +2.8%     -0.1%      0.00      0.00
          event          +2.6%     -0.0%     -4.9%     -4.4%
         exp3_8          +2.8%     +0.0%     -4.5%     -9.5%
         expert          +2.7%     +0.3%      0.00      0.00
            fem          -2.0%     +0.6%      0.04      0.04
            fft          -6.0%     +1.8%      0.05      0.06
           fft2          -4.8%     +2.7%      0.13      0.14
       fibheaps          +2.6%     -0.6%      0.05      0.05
           fish          +4.1%     +0.0%      0.03      0.04
          fluid          -2.1%     -0.2%      0.01      0.01
         fulsom          -4.8%     +9.2%     +9.1%     +8.4%
         gamteb          -7.1%     -1.3%      0.10      0.11
            gcd          +2.7%     +0.0%      0.05      0.05
    gen_regexps          +3.9%     -0.0%      0.00      0.00
         genfft          +2.7%     -0.1%      0.05      0.06
             gg          -2.7%     -0.1%      0.02      0.02
           grep          +3.2%     -0.0%      0.00      0.00
         hidden          -0.5%     +0.0%    -11.9%    -13.3%
            hpg          -3.0%     -1.8%     +0.0%     -2.4%
            ida          +2.6%     -1.2%      0.17     -9.0%
          infer          +1.7%     -0.8%      0.08      0.09
        integer          +2.5%     -0.0%     -2.6%     -2.2%
      integrate          -5.0%     +0.0%     -1.3%     -2.9%
        knights          +4.3%     -1.5%      0.01      0.01
           lcss          +2.5%     -0.1%     -7.5%     -9.4%
           life          +4.2%     +0.0%     -3.1%     -3.3%
           lift          +2.4%     -3.2%      0.00      0.00
      listcompr          +4.0%     -1.6%      0.16      0.17
       listcopy          +4.0%     -1.4%      0.17      0.18
       maillist          +4.1%     +0.1%      0.09      0.14
         mandel          +2.9%     +0.0%      0.11      0.12
        mandel2          +4.7%     +0.0%      0.01      0.01
        minimax          +3.8%     -0.0%      0.00      0.00
        mkhprog          +3.2%     -4.2%      0.00      0.00
     multiplier          +2.5%     -0.4%     +0.7%     -1.3%
       nucleic2          -9.3%     +0.0%      0.10      0.10
           para          +2.9%     +0.1%     -0.7%     -1.2%
      paraffins         -10.4%     +0.0%      0.20     -1.9%
         parser          +3.1%     -0.0%      0.05      0.05
        parstof          +1.9%     -0.0%      0.00      0.01
            pic          -2.8%     -0.8%      0.01      0.02
          power          +2.1%     +0.1%     -8.5%     -9.0%
         pretty         -12.7%     +0.1%      0.00      0.00
         primes          +2.8%     +0.0%      0.11      0.11
      primetest          +2.5%     -0.0%     -2.1%     -3.1%
         prolog          +3.2%     -7.2%      0.00      0.00
         puzzle          +4.1%     +0.0%     -3.5%     -8.0%
         queens          +2.8%     +0.0%      0.03      0.03
        reptile          +2.2%     -2.2%      0.02      0.02
        rewrite          +3.1%    +10.9%      0.03      0.03
           rfib          -5.2%     +0.2%      0.03      0.03
            rsa          +2.6%     +0.0%      0.05      0.06
            scc          +4.6%     +0.4%      0.00      0.00
          sched          +2.7%     +0.1%      0.03      0.03
            scs          -2.6%     -0.9%     -9.6%    -11.6%
         simple          -4.0%     +0.4%    -14.6%    -14.9%
          solid          -5.6%     -0.6%     -9.3%    -14.3%
        sorting          +3.8%     +0.0%      0.00      0.00
         sphere          -3.6%     +8.5%      0.15      0.16
         symalg          -1.3%     +0.2%      0.03      0.03
            tak          +2.7%     +0.0%      0.02      0.02
      transform          +2.0%     -2.9%     -8.0%     -8.8%
       treejoin          +3.1%     +0.0%    -17.5%    -17.8%
      typecheck          +2.9%     -0.3%     -4.6%     -6.6%
        veritas          +3.9%     -0.3%      0.00      0.00
           wang          -6.2%     +0.0%      0.18     -9.8%
      wave4main         -10.3%     +2.6%     -2.1%     -2.3%
   wheel-sieve1          +2.7%     -0.0%     +0.3%     -0.6%
   wheel-sieve2          +2.7%     +0.0%     -3.7%     -7.5%
           x2n1          -4.1%     +0.1%      0.03      0.04
--------------------------------------------------------------------------------
            Min         -12.7%    -14.5%    -17.5%    -17.8%
            Max          +4.7%    +10.9%     +9.1%     +8.4%
 Geometric Mean          +0.9%     -0.1%     -5.6%     -7.3%

This patch adds an optional CONLIKE modifier to INLINE/NOINLINE pragmas, 
   {-# NOINLINE CONLIKE [1] f #-}
The effect is to allow applications of 'f' to be expanded in a potential
rule match.  Example
  {-# RULE "r/f" forall v. r (f v) = f (v+1) #-}

Consider the term
     let x = f v in ..x...x...(r x)...
Normally the (r x) would not match the rule, because GHC would be scared
about duplicating the redex (f v). However the CONLIKE modifier says to
treat 'f' like a constructor in this situation, and "look through" the
unfolding for x.  So (r x) fires, yielding (f (v+1)).

The main changes are:
  - Syntax

  - The inlinePragInfo field of an IdInfo has a RuleMatchInfo
    component, which records whether or not the Id is CONLIKE.
    Of course, this needs to be serialised in interface files too.

  - The occurrence analyser (OccAnal) and simplifier (Simplify) treat
    CONLIKE thing like constructors, by ANF-ing them

  - New function coreUtils.exprIsExpandable is like exprIsCheap, but
    additionally spots applications of CONLIKE functions

  - A CoreUnfolding has a field that caches exprIsExpandable

  - The rule matcher consults this field.  See 
    Note [Expanding variables] in Rules.lhs.

On the way I fixed a lurking variable bug in the way variables are
expanded.  See Note [Do not expand locally-bound variables] in
Rule.lhs.  I also did a bit of reformatting and refactoring in
Rules.lhs, so the module has more lines changed than are really
different.

This biggish patch addresses Trac #2670.  The main effect is to make
record selectors into ordinary functions, whose unfoldings appear in
interface files, in contrast to their previous existence as magic
"implicit Ids".  This means that the usual machinery of optimisation,
analysis, and inlining applies to them, which was failing before when
the selector was somewhat complicated.  (Which it can be when
strictness annotations, unboxing annotations, and GADTs are involved.)

The change involves the following points

* Changes in Var.lhs to the representation of Var.  Now a LocalId can
  have an IdDetails as well as a GlobalId.  In particular, the
  information that an Id is a record selector is kept in the
  IdDetails.  While compiling the current module, the record selector
  *must* be a LocalId, so that it participates properly in compilation
  (free variables etc).

  This led me to change the (hidden) representation of Var, so that there
  is now only one constructor for Id, not two.

* The IdDetails is persisted into interface files, so that an
  importing module can see which Ids are records selectors.

* In TcTyClDecls, we generate the record-selector bindings in renamed,
  but not typechecked form.  In this way, we can get the typechecker
  to add all the types and so on, which is jolly helpful especially
  when GADTs or type families are involved.  Just like derived
  instance declarations.

  This is the big new chunk of 180 lines of code (much of which is
  commentary).  A call to the same function, mkAuxBinds, is needed in
  TcInstDcls for associated types.

* The typechecker therefore has to pin the correct IdDetails on to 
  the record selector, when it typechecks it.  There was a neat way
  to do this, by adding a new sort of signature to HsBinds.Sig, namely
  IdSig.  This contains an Id (with the correct Name, Type, and IdDetails);
  the type checker uses it as the binder for the final binding.  This
  worked out rather easily.

* Record selectors are no longer "implicit ids", which entails changes to
     IfaceSyn.ifaceDeclSubBndrs
     HscTypes.implicitTyThings
     TidyPgm.getImplicitBinds
  (These three functions must agree.)

* MkId.mkRecordSelectorId is deleted entirely, some 300+ lines (incl
  comments) of very error prone code.  Happy days.

* A TyCon no longer contains the list of record selectors: 
  algTcSelIds is gone

The renamer is unaffected, including the way that import and export of
record selectors is handled.

Other small things

* IfaceSyn.ifaceDeclSubBndrs had a fragile test for whether a data
  constructor had a wrapper.  I've replaced that with an explicit flag
  in the interface file. More robust I hope.

* I renamed isIdVar to isId, which touched a few otherwise-unrelated files.

rolling back:

Fri Dec  5 16:54:00 GMT 2008  simonpj@microsoft.com
  * Completely new treatment of INLINE pragmas (big patch)
  
  This is a major patch, which changes the way INLINE pragmas work.
  Although lots of files are touched, the net is only +21 lines of
  code -- and I bet that most of those are comments!
  
  HEADS UP: interface file format has changed, so you'll need to
  recompile everything.
  
  There is not much effect on overall performance for nofib, 
  probably because those programs don't make heavy use of INLINE pragmas.
  
          Program           Size    Allocs   Runtime   Elapsed
              Min         -11.3%     -6.9%     -9.2%     -8.2%
              Max          -0.1%     +4.6%     +7.5%     +8.9%
   Geometric Mean          -2.2%     -0.2%     -1.0%     -0.8%
  
  (The +4.6% for on allocs is cichelli; see other patch relating to
  -fpass-case-bndr-to-join-points.)
  
  The old INLINE system
  ~~~~~~~~~~~~~~~~~~~~~
  The old system worked like this. A function with an INLINE pragam
  got a right-hand side which looked like
       f = __inline_me__ (\xy. e)
  The __inline_me__ part was an InlineNote, and was treated specially
  in various ways.  Notably, the simplifier didn't inline inside an
  __inline_me__ note.  
  
  As a result, the code for f itself was pretty crappy. That matters
  if you say (map f xs), because then you execute the code for f,
  rather than inlining a copy at the call site.
  
  The new story: InlineRules
  ~~~~~~~~~~~~~~~~~~~~~~~~~~
  The new system removes the InlineMe Note altogether.  Instead there
  is a new constructor InlineRule in CoreSyn.Unfolding.  This is a 
  bit like a RULE, in that it remembers the template to be inlined inside
  the InlineRule.  No simplification or inlining is done on an InlineRule,
  just like RULEs.  
  
  An Id can have an InlineRule *or* a CoreUnfolding (since these are two
  constructors from Unfolding). The simplifier treats them differently:
  
    - An InlineRule is has the substitution applied (like RULES) but 
      is otherwise left undisturbed.
  
    - A CoreUnfolding is updated with the new RHS of the definition,
      on each iteration of the simplifier.
  
  An InlineRule fires regardless of size, but *only* when the function
  is applied to enough arguments.  The "arity" of the rule is specified
  (by the programmer) as the number of args on the LHS of the "=".  So
  it makes a difference whether you say
    	{-# INLINE f #-}
  	f x = \y -> e     or     f x y = e
  This is one of the big new features that InlineRule gives us, and it
  is one that Roman really wanted.
  
  In contrast, a CoreUnfolding can fire when it is applied to fewer
  args than than the function has lambdas, provided the result is small
  enough.
  
  
  Consequential stuff
  ~~~~~~~~~~~~~~~~~~~
  * A 'wrapper' no longer has a WrapperInfo in the IdInfo.  Instead,
    the InlineRule has a field identifying wrappers.
  
  * Of course, IfaceSyn and interface serialisation changes appropriately.
  
  * Making implication constraints inline nicely was a bit fiddly. In
    the end I added a var_inline field to HsBInd.VarBind, which is why
    this patch affects the type checker slightly
  
  * I made some changes to the way in which eta expansion happens in
    CorePrep, mainly to ensure that *arguments* that become let-bound
    are also eta-expanded.  I'm still not too happy with the clarity
    and robustness fo the result.
  
  * We now complain if the programmer gives an INLINE pragma for
    a recursive function (prevsiously we just ignored it).  Reason for
    change: we don't want an InlineRule on a LoopBreaker, because then
    we'd have to check for loop-breaker-hood at occurrence sites (which
    isn't currenlty done).  Some tests need changing as a result.
  
  This patch has been in my tree for quite a while, so there are
  probably some other minor changes.
  

    M ./compiler/basicTypes/Id.lhs -11
    M ./compiler/basicTypes/IdInfo.lhs -82
    M ./compiler/basicTypes/MkId.lhs -2 +2
    M ./compiler/coreSyn/CoreFVs.lhs -2 +25
    M ./compiler/coreSyn/CoreLint.lhs -5 +1
    M ./compiler/coreSyn/CorePrep.lhs -59 +53
    M ./compiler/coreSyn/CoreSubst.lhs -22 +31
    M ./compiler/coreSyn/CoreSyn.lhs -66 +92
    M ./compiler/coreSyn/CoreUnfold.lhs -112 +112
    M ./compiler/coreSyn/CoreUtils.lhs -185 +184
    M ./compiler/coreSyn/MkExternalCore.lhs -1
    M ./compiler/coreSyn/PprCore.lhs -4 +40
    M ./compiler/deSugar/DsBinds.lhs -70 +118
    M ./compiler/deSugar/DsForeign.lhs -2 +4
    M ./compiler/deSugar/DsMeta.hs -4 +3
    M ./compiler/hsSyn/HsBinds.lhs -3 +3
    M ./compiler/hsSyn/HsUtils.lhs -2 +7
    M ./compiler/iface/BinIface.hs -11 +25
    M ./compiler/iface/IfaceSyn.lhs -13 +21
    M ./compiler/iface/MkIface.lhs -24 +19
    M ./compiler/iface/TcIface.lhs -29 +23
    M ./compiler/main/TidyPgm.lhs -55 +49
    M ./compiler/parser/ParserCore.y -5 +6
    M ./compiler/simplCore/CSE.lhs -2 +1
    M ./compiler/simplCore/FloatIn.lhs -6 +1
    M ./compiler/simplCore/FloatOut.lhs -23
    M ./compiler/simplCore/OccurAnal.lhs -36 +5
    M ./compiler/simplCore/SetLevels.lhs -59 +54
    M ./compiler/simplCore/SimplCore.lhs -48 +52
    M ./compiler/simplCore/SimplEnv.lhs -26 +22
    M ./compiler/simplCore/SimplUtils.lhs -28 +4
    M ./compiler/simplCore/Simplify.lhs -91 +109
    M ./compiler/specialise/Specialise.lhs -15 +18
    M ./compiler/stranal/WorkWrap.lhs -14 +11
    M ./compiler/stranal/WwLib.lhs -2 +2
    M ./compiler/typecheck/Inst.lhs -1 +3
    M ./compiler/typecheck/TcBinds.lhs -17 +27
    M ./compiler/typecheck/TcClassDcl.lhs -1 +2
    M ./compiler/typecheck/TcExpr.lhs -4 +6
    M ./compiler/typecheck/TcForeign.lhs -1 +1
    M ./compiler/typecheck/TcGenDeriv.lhs -14 +13
    M ./compiler/typecheck/TcHsSyn.lhs -3 +2
    M ./compiler/typecheck/TcInstDcls.lhs -5 +4
    M ./compiler/typecheck/TcRnDriver.lhs -2 +11
    M ./compiler/typecheck/TcSimplify.lhs -10 +17
    M ./compiler/vectorise/VectType.hs +7

Mon Dec  8 12:43:10 GMT 2008  simonpj@microsoft.com
  * White space only

    M ./compiler/simplCore/Simplify.lhs -2

Mon Dec  8 12:48:40 GMT 2008  simonpj@microsoft.com
  * Move simpleOptExpr from CoreUnfold to CoreSubst

    M ./compiler/coreSyn/CoreSubst.lhs -1 +87
    M ./compiler/coreSyn/CoreUnfold.lhs -72 +1

Mon Dec  8 17:30:18 GMT 2008  simonpj@microsoft.com
  * Use CoreSubst.simpleOptExpr in place of the ad-hoc simpleSubst (reduces code too)

    M ./compiler/deSugar/DsBinds.lhs -50 +16

Tue Dec  9 17:03:02 GMT 2008  simonpj@microsoft.com
  * Fix Trac #2861: bogus eta expansion
  
  Urghlhl!  I "tided up" the treatment of the "state hack" in CoreUtils, but
  missed an unexpected interaction with the way that a bottoming function
  simply swallows excess arguments.  There's a long
       Note [State hack and bottoming functions]
  to explain (which accounts for most of the new lines of code).
  

    M ./compiler/coreSyn/CoreUtils.lhs -16 +53

Mon Dec 15 10:02:21 GMT 2008  Simon Marlow <marlowsd@gmail.com>
  * Revert CorePrep part of "Completely new treatment of INLINE pragmas..."
  
  The original patch said:
  
  * I made some changes to the way in which eta expansion happens in
    CorePrep, mainly to ensure that *arguments* that become let-bound
    are also eta-expanded.  I'm still not too happy with the clarity
    and robustness fo the result.
    
  Unfortunately this change apparently broke some invariants that were
  relied on elsewhere, and in particular lead to panics when compiling
  with profiling on.
  
  Will re-investigate in the new year.

    M ./compiler/coreSyn/CorePrep.lhs -53 +58
    M ./configure.ac -1 +1

Mon Dec 15 12:28:51 GMT 2008  Simon Marlow <marlowsd@gmail.com>
  * revert accidental change to configure.ac

    M ./configure.ac -1 +1

This is a major patch, which changes the way INLINE pragmas work.
Although lots of files are touched, the net is only +21 lines of
code -- and I bet that most of those are comments!

HEADS UP: interface file format has changed, so you'll need to
recompile everything.

There is not much effect on overall performance for nofib, 
probably because those programs don't make heavy use of INLINE pragmas.

        Program           Size    Allocs   Runtime   Elapsed
            Min         -11.3%     -6.9%     -9.2%     -8.2%
            Max          -0.1%     +4.6%     +7.5%     +8.9%
 Geometric Mean          -2.2%     -0.2%     -1.0%     -0.8%

(The +4.6% for on allocs is cichelli; see other patch relating to
-fpass-case-bndr-to-join-points.)

The old INLINE system
~~~~~~~~~~~~~~~~~~~~~
The old system worked like this. A function with an INLINE pragam
got a right-hand side which looked like
     f = __inline_me__ (\xy. e)
The __inline_me__ part was an InlineNote, and was treated specially
in various ways.  Notably, the simplifier didn't inline inside an
__inline_me__ note.  

As a result, the code for f itself was pretty crappy. That matters
if you say (map f xs), because then you execute the code for f,
rather than inlining a copy at the call site.

The new story: InlineRules
~~~~~~~~~~~~~~~~~~~~~~~~~~
The new system removes the InlineMe Note altogether.  Instead there
is a new constructor InlineRule in CoreSyn.Unfolding.  This is a 
bit like a RULE, in that it remembers the template to be inlined inside
the InlineRule.  No simplification or inlining is done on an InlineRule,
just like RULEs.  

An Id can have an InlineRule *or* a CoreUnfolding (since these are two
constructors from Unfolding). The simplifier treats them differently:

  - An InlineRule is has the substitution applied (like RULES) but 
    is otherwise left undisturbed.

  - A CoreUnfolding is updated with the new RHS of the definition,
    on each iteration of the simplifier.

An InlineRule fires regardless of size, but *only* when the function
is applied to enough arguments.  The "arity" of the rule is specified
(by the programmer) as the number of args on the LHS of the "=".  So
it makes a difference whether you say
  	{-# INLINE f #-}
	f x = \y -> e     or     f x y = e
This is one of the big new features that InlineRule gives us, and it
is one that Roman really wanted.

In contrast, a CoreUnfolding can fire when it is applied to fewer
args than than the function has lambdas, provided the result is small
enough.


Consequential stuff
~~~~~~~~~~~~~~~~~~~
* A 'wrapper' no longer has a WrapperInfo in the IdInfo.  Instead,
  the InlineRule has a field identifying wrappers.

* Of course, IfaceSyn and interface serialisation changes appropriately.

* Making implication constraints inline nicely was a bit fiddly. In
  the end I added a var_inline field to HsBInd.VarBind, which is why
  this patch affects the type checker slightly

* I made some changes to the way in which eta expansion happens in
  CorePrep, mainly to ensure that *arguments* that become let-bound
  are also eta-expanded.  I'm still not too happy with the clarity
  and robustness fo the result.

* We now complain if the programmer gives an INLINE pragma for
  a recursive function (prevsiously we just ignored it).  Reason for
  change: we don't want an InlineRule on a LoopBreaker, because then
  we'd have to check for loop-breaker-hood at occurrence sites (which
  isn't currenlty done).  Some tests need changing as a result.

This patch has been in my tree for quite a while, so there are
probably some other minor changes.

This patch, written by Max Bolingbroke,  does two things

1.  It adds a new CoreM monad (defined in simplCore/CoreMonad),
    which is used as the top-level monad for all the Core-to-Core
    transformations (starting at SimplCore).  It supports
       * I/O (for debug printing)
       * Unique supply
       * Statistics gathering
       * Access to the HscEnv, RuleBase, Annotations, Module
    The patch therefore refactors the top "skin" of every Core-to-Core
    pass, but does not change their functionality.

2.  It adds a completely new facility to GHC: Core "annotations".
    The idea is that you can say
       {#- ANN foo (Just "Hello") #-}
    which adds the annotation (Just "Hello") to the top level function
    foo.  These annotations can be looked up in any Core-to-Core pass,
    and are persisted into interface files.  (Hence a Core-to-Core pass
    can also query the annotations of imported things.)  Furthermore,
    a Core-to-Core pass can add new annotations (eg strictness info)
    of its own, which can be queried by importing modules.

The design of the annotation system is somewhat in flux.  It's
designed to work with the (upcoming) dynamic plug-ins mechanism,
but is meanwhile independently useful.

Do not merge to 6.10!