GitLab

runRW# isn't inlined until CorePrep, so it's good to expose its
strictness.  Moreover, if we don't we can get obscure failures
in coreToStg; see Note [runRW arg] in CorePrep.

This fixes Trac #11291, and makes DfltProb1 compile with -O
always in order to expose it more vigorously

Since GHC 8.1/8.2 only needs to be bootstrap-able by GHC 7.10 and
GHC 8.0 (and GHC 8.2), we can now finally drop all that pre-AMP
compatibility CPP-mess for good!

Reviewers: austin, goldfire, bgamari

Subscribers: goldfire, thomie, erikd

Differential Revision: https://phabricator.haskell.org/D1724

Instead of substituting, just augment the environment.
Less code, more efficient.

And the previous version had a bogus in-scope set which
triggered a WARNING

This implements the ideas originally put forward in
"System FC with Explicit Kind Equality" (ICFP'13).

There are several noteworthy changes with this patch:
 * We now have casts in types. These change the kind
   of a type. See new constructor `CastTy`.

 * All types and all constructors can be promoted.
   This includes GADT constructors. GADT pattern matches
   take place in type family equations. In Core,
   types can now be applied to coercions via the
   `CoercionTy` constructor.

 * Coercions can now be heterogeneous, relating types
   of different kinds. A coercion proving `t1 :: k1 ~ t2 :: k2`
   proves both that `t1` and `t2` are the same and also that
   `k1` and `k2` are the same.

 * The `Coercion` type has been significantly enhanced.
   The documentation in `docs/core-spec/core-spec.pdf` reflects
   the new reality.

 * The type of `*` is now `*`. No more `BOX`.

 * Users can write explicit kind variables in their code,
   anywhere they can write type variables. For backward compatibility,
   automatic inference of kind-variable binding is still permitted.

 * The new extension `TypeInType` turns on the new user-facing
   features.

 * Type families and synonyms are now promoted to kinds. This causes
   trouble with parsing `*`, leading to the somewhat awkward new
   `HsAppsTy` constructor for `HsType`. This is dispatched with in
   the renamer, where the kind `*` can be told apart from a
   type-level multiplication operator. Without `-XTypeInType` the
   old behavior persists. With `-XTypeInType`, you need to import
   `Data.Kind` to get `*`, also known as `Type`.

 * The kind-checking algorithms in TcHsType have been significantly
   rewritten to allow for enhanced kinds.

 * The new features are still quite experimental and may be in flux.

 * TODO: Several open tickets: #11195, #11196, #11197, #11198, #11203.

 * TODO: Update user manual.

Tickets addressed: #9017, #9173, #7961, #10524, #8566, #11142.
Updates Haddock submodule.

As you'll see from Trac #11155, the code generator was confused
by a binding let x = y in ....   Why did that happen? Because of
a (case y of {}) expression on the RHS.

The right thing is just to expand what a "trivial" expression is.

See Note [Empty case is trivial] in CoreUtils.

We now only strip block information from DebugBlocks when compiling with
`-g1`, intended to be used when only minimal debug information is
desired. `-g2` is assumed when `-g` is passed without any integer
argument.

Differential Revision: https://phabricator.haskell.org/D1281

Test Plan: validate

Reviewers: goldfire, erikd, rwbarton, simonpj, austin, simonmar, hvr

Reviewed By: simonpj

Subscribers: simonmar, thomie

Differential Revision: https://phabricator.haskell.org/D1103

GHC Trac Issues: #10678

Comes with Haddock submodule update.
Signed-off-by: Edward Z. Yang <ezyang@cs.stanford.edu>

Test Plan: validate

Reviewers: simonpj, austin, bgamari

Reviewed By: bgamari

Subscribers: thomie

Differential Revision: https://phabricator.haskell.org/D1319

Reviewers: austin, bgamari

Reviewed By: bgamari

Subscribers: thomie

Differential Revision: https://phabricator.haskell.org/D1190

This localises the (revolting) initTcForLookup function, exposing
instead the more civilised interface for lookupGlobal

This completes what c774b28f (#9281)
started.  `integer-gmp-1.0` was added as an additional
`libraries/integer-gmp2` folder while retaining the ability to configure
GHC w/ the old `integer-gmp-0.5` to have a way back, and or the ability
to easily switch between old/new `integer-gmp` for benchmark/debugging
purposes.

This commit removes the old `libraries/integer-gmp` folder and moves
`libraries/integer-gmp2` into its place, while removing any mentions of
"gmp2" as well as the to support two different `integer-gmp` packages in
GHC's source-tree.

Reviewed By: austin

Differential Revision: https://phabricator.haskell.org/D769

Summary: It looks like during .lhs -> .hs switch the comments were not updated. So doing exactly that.

Reviewers: austin, jstolarek, hvr, goldfire

Reviewed By: austin, jstolarek

Subscribers: thomie, goldfire

Differential Revision: https://phabricator.haskell.org/D621

GHC Trac Issues: #9986

See Note [Disgusting computation of CafRefs] in TidyPgm.

Also affects CoreUtils.rhsIsStatic.

The real solution here is to compute CAF and arity information
from the STG-program, and feed it back to tidied program for
the interface file and later GHCi clients.  A battle for another
day.

But at least this commit reduces the number of gratuitous CAFs, and
hence SRT entries.  And kills off a batch of ASSERT failures.

This is basically just about continuing maintaining source notes after
the Core stage. Unfortunately, this is more involved as it might seem,
as there are more restrictions on where ticks are allowed to show up.

Notes:

* We replace the StgTick / StgSCC constructors with a unified StgTick
  that can carry any tickish.

* For handling constructor or lambda applications, we generally float
  ticks out.

* Note that thanks to the NonLam placement, we know that source notes
  can never appear on lambdas. This means that as long as we are
  careful to always use mkTick, we will never violate CorePrep
  invariants.

* This is however not automatically true for eta expansion, which
  needs to somewhat awkwardly strip, then re-tick the expression in
  question.

* Where CorePrep floats out lets, we make sure to wrap them in the
  same spirit as FloatOut.

* Detecting selector thunks becomes a bit more involved, as we can run
  into ticks at multiple points.

(From Phabricator D169)

This patch introduces "SourceNote" tickishs that link Core to the
source code that generated it. The idea is to retain these source code
links throughout code transformations so we can eventually relate
object code all the way back to the original source (which we can,
say, encode as DWARF information to allow debugging).  We generate
these SourceNotes like other tickshs in the desugaring phase. The
activating command line flag is "-g", consistent with the flag other
compilers use to decide DWARF generation.

Keeping ticks from getting into the way of Core transformations is
tricky, but doable. The changes in this patch produce identical Core
in all cases I tested -- which at this point is GHC, all libraries and
nofib. Also note that this pass creates *lots* of tick nodes, which we
reduce somewhat by removing duplicated and overlapping source
ticks. This will still cause significant Tick "clumps" - a possible
future optimization could be to make Tick carry a list of Tickishs
instead of one at a time.

(From Phabricator D169)

There should be no bindings in this module for a GlobalId;
except after CoreTidy, when top-level bindings are globalised.

To check for this, I had to make the CoreToDo pass part of the
environment that Core Lint caries.  But CoreToDo is defined in
CoreMonad, which (before this patch) called CoreLint.

So I had to do quite a bit of refactoring, moving some
lint-invoking code into CoreLint itself.  Crucially, I also
more tcLookupImported_maybe, importDecl, and checkwiredInTyCon
from TcIface (which use CoreLint) to LoadIface (which doesn't).
This is probably better structure anyway.

So most of this patch is refactoring. The actual check for
GlobalIds is in CoreLint.lintAndScopeId

Signed-off-by: Austin Seipp <austin@well-typed.com>

This is done as a separate `integer-gmp2` backend library because it
turned out to become a complete rewrite from scratch.

Due to the different (over)allocation scheme and potentially different
accounting (via the new `{shrink,resize}MutableByteArray#` primitives),
some of the nofib benchmarks actually results in increased allocation
numbers (but not necessarily an increase in runtime!).  I believe the
allocation numbers could improve if `{resize,shrink}MutableByteArray#`
could be optimised to reallocate in-place more efficiently.

Here are the more apparent changes in the latest nofib comparision
between `integer-gmp` and `integer-gmp2`:

  ------------------------------------------------------------------
          Program     Size    Allocs   Runtime   Elapsed  TotalMem
  ------------------------------------------------------------------
              ...
       bernouilli    +1.6%    +15.3%     0.132     0.132      0.0%
              ...
     cryptarithm1    -2.2%      0.0%     -9.7%     -9.7%      0.0%
              ...
            fasta    -0.7%     -0.0%    +10.9%    +10.9%      0.0%
              ...
            kahan    +0.6%    +38.9%     0.169     0.169      0.0%
              ...
             lcss    -0.7%     -0.0%     -6.4%     -6.4%      0.0%
              ...
           mandel    +1.6%    +33.6%     0.049     0.049      0.0%
              ...
         pidigits    +0.8%     +8.5%     +3.9%     +3.9%      0.0%
            power    +1.4%    -23.8%    -18.6%    -18.6%    -16.7%
              ...
        primetest    +1.3%    +50.1%     0.085     0.085      0.0%
              ...
              rsa    +1.6%    +53.4%     0.026     0.026      0.0%
              ...
              scs    +1.2%     +6.6%     +6.5%     +6.6%    +14.3%
              ...
           symalg    +1.0%     +9.5%     0.010     0.010      0.0%
              ...
        transform    -0.6%     -0.0%     -5.9%     -5.9%      0.0%
              ...
  ------------------------------------------------------------------
              Min    -2.3%    -23.8%    -18.6%    -18.6%    -16.7%
              Max    +1.6%    +53.4%    +10.9%    +10.9%    +14.3%
   Geometric Mean    -0.3%     +1.9%     -0.8%     -0.8%     +0.0%

(see P35 / https://phabricator.haskell.org/P35 for full report)

By default, `INTEGER_LIBRARY=integer-gmp2` is active now, which results
in the package `integer-gmp-1.0.0.0` being registered in the package db.
The previous `integer-gmp-0.5.1.0` can be restored by setting
`INTEGER_LIBRARY=integer-gmp` (but will probably be removed altogether
for GHC 7.12). In-tree GMP support has been stolen from the old
`integer-gmp` (while unpatching the custom memory-allocators, as well as
forcing `-fPIC`)

A minor hack to `ghc-cabal` was necessary in order to support two different
`integer-gmp` packages (in different folders) with the same package key.

There will be a couple of follow-up commits re-implementing some features
that were dropped to keep D82 minimal, as well as further
clean-ups/improvements.

More information can be found via #9281 and
https://ghc.haskell.org/trac/ghc/wiki/Design/IntegerGmp2

Reviewed By: austin, rwbarton, simonmar

Differential Revision: https://phabricator.haskell.org/D82

I forget all the details, but I spent some time trying to
understand the current setup, and tried to simplify it a bit

Integer is currently a wired-in type for integer-gmp. This requires
replicating its inner structure in `TysWiredIn`, which makes it much
harder to change Integer to a more complex representation (as
e.g. needed for implementing #9281)

This commit stops `Integer` being a wired-in type, and makes it
known-key type instead, thereby simplifying code notably.

Reviewed By: austin

Differential Revision: https://phabricator.haskell.org/D351

Summary:
Previously, both Cabal and GHC defined the type PackageId, and we expected
them to be roughly equivalent (but represented differently).  This refactoring
separates these two notions.

A package ID is a user-visible identifier; it's the thing you write in a
Cabal file, e.g. containers-0.9.  The components of this ID are semantically
meaningful, and decompose into a package name and a package vrsion.

A package key is an opaque identifier used by GHC to generate linking symbols.
Presently, it just consists of a package name and a package version, but
pursuant to #9265 we are planning to extend it to record other information.
Within a single executable, it uniquely identifies a package.  It is *not* an
InstalledPackageId, as the choice of a package key affects the ABI of a package
(whereas an InstalledPackageId is computed after compilation.)  Cabal computes
a package key for the package and passes it to GHC using -package-name (now
*extremely* misnamed).

As an added bonus, we don't have to worry about shadowing anymore.

As a follow on, we should introduce -current-package-key having the same role as
-package-name, and deprecate the old flag.  This commit is just renaming.

The haddock submodule needed to be updated.
Signed-off-by: Edward Z. Yang <ezyang@cs.stanford.edu>

Test Plan: validate

Reviewers: simonpj, simonmar, hvr, austin

Subscribers: simonmar, relrod, carter

Differential Revision: https://phabricator.haskell.org/D79

Conflicts:
	compiler/main/HscTypes.lhs
	compiler/main/Packages.lhs
	utils/haddock

This way, the Ids for the matchers/wrappers are reused by importing
modules, and thus unfoldings are kept.

Also updates haddock submodule to accomodate tweaks in PatSyn representation

In some cases, the layout of the LANGUAGE/OPTIONS_GHC lines has been
reorganized, while following the convention, to

- place `{-# LANGUAGE #-}` pragmas at the top of the source file, before
  any `{-# OPTIONS_GHC #-}`-lines.

- Moreover, if the list of language extensions fit into a single
  `{-# LANGUAGE ... -#}`-line (shorter than 80 characters), keep it on one
  line. Otherwise split into `{-# LANGUAGE ... -#}`-lines for each
  individual language extension. In both cases, try to keep the
  enumeration alphabetically ordered.
  (The latter layout is preferable as it's more diff-friendly)

While at it, this also replaces obsolete `{-# OPTIONS ... #-}` pragma
occurences by `{-# OPTIONS_GHC ... #-}` pragmas.

The original problem was that we weren't bringing varaibles bound in the
interactive context into scope before Linting the result of a top-level
declaration in GHCi.  (We were doing this for expressions.)

Moreover I found that we weren't Linting the result of desugaring
a GHCi expression, which we really should be doing.

It took me a bit of time to unravel all this, and I did some refactoring
to make it easier next time.

  * CoreMonad contains the Lint wrappers that get the right
    environments into place.  It always had endPass and lintPassResult
    (which Lints bindings), but now it has lintInteractiveExpr.

  * Both use a common function CoreMonad.interactiveInScope to find
    those in-scope variables.

Quite a bit of knock-on effects from this, but nothing exciting.

This major patch implements the cardinality analysis described
in our paper "Higher order cardinality analysis". It is joint
work with Ilya Sergey and Dimitrios Vytiniotis.

The basic is augment the absence-analysis part of the demand
analyser so that it can tell when something is used
	 never
	 at most once
 	 some other way

The "at most once" information is used
    a) to enable transformations, and
       in particular to identify one-shot lambdas
    b) to allow updates on thunks to be omitted.

There are two new flags, mainly there so you can do performance
comparisons:
    -fkill-absence   stops GHC doing absence analysis at all
    -fkill-one-shot  stops GHC spotting one-shot lambdas
                     and single-entry thunks

The big changes are:

* The Demand type is substantially refactored.  In particular
  the UseDmd is factored as follows
      data UseDmd
        = UCall Count UseDmd
        | UProd [MaybeUsed]
        | UHead
        | Used

      data MaybeUsed = Abs | Use Count UseDmd

      data Count = One | Many

  Notice that UCall recurses straight to UseDmd, whereas
  UProd goes via MaybeUsed.

  The "Count" embodies the "at most once" or "many" idea.

* The demand analyser itself was refactored a lot

* The previously ad-hoc stuff in the occurrence analyser for foldr and
  build goes away entirely.  Before if we had build (\cn -> ...x... )
  then the "\cn" was hackily made one-shot (by spotting 'build' as
  special.  That's essential to allow x to be inlined.  Now the
  occurrence analyser propagates info gotten from 'build's stricness
  signature (so build isn't special); and that strictness sig is
  in turn derived entirely automatically.  Much nicer!

* The ticky stuff is improved to count single-entry thunks separately.

One shortcoming is that there is no DEBUG way to spot if an
allegedly-single-entry thunk is acually entered more than once.  It
would not be hard to generate a bit of code to check for this, and it
would be reassuring.  But it's fiddly and I have not done it.

Despite all this fuss, the performance numbers are rather under-whelming.
See the paper for more discussion.

       nucleic2          -0.8%    -10.9%      0.10      0.10     +0.0%
         sphere          -0.7%     -1.5%      0.08      0.08     +0.0%
--------------------------------------------------------------------------------
            Min          -4.7%    -10.9%     -9.3%     -9.3%    -50.0%
            Max          -0.4%     +0.5%     +2.2%     +2.3%     +7.4%
 Geometric Mean          -0.8%     -0.2%     -1.3%     -1.3%     -1.8%

I don't quite know how much credence to place in the runtime changes,
but movement seems generally in the right direction.

We were trying to look up the mkInteger Id before we'd compiled
the Integer modules. I'm not sure why this never showed up with
integer-gmp; possibly we just always got lucky with the build
order.

This patch is the result of Ilya Sergey's internship at MSR.  It
constitutes a thorough overhaul and simplification of the demand
analyser.  It makes a solid foundation on which we can now build.
Main changes are

* Instead of having one combined type for Demand, a Demand is
   now a pair (JointDmd) of
      - a StrDmd and
      - an AbsDmd.
   This allows strictness and absence to be though about quite
   orthogonally, and greatly reduces brain melt-down.

* Similarly in the DmdResult type, it's a pair of
     - a PureResult (indicating only divergence/non-divergence)
     - a CPRResult (which deals only with the CPR property

* In IdInfo, the
    strictnessInfo field contains a StrictSig, not a Maybe StrictSig
    demandInfo     field contains a Demand, not a Maybe Demand
  We don't need Nothing (to indicate no strictness/demand info)
  any more; topSig/topDmd will do.

* Remove "boxity" analysis entirely.  This was an attempt to
  avoid "reboxing", but it added complexity, is extremely
  ad-hoc, and makes very little difference in practice.

* Remove the "unboxing strategy" computation. This was an an
  attempt to ensure that a worker didn't get zillions of
  arguments by unboxing big tuples.  But in fact removing it
  DRAMATICALLY reduces allocation in an inner loop of the
  I/O library (where the threshold argument-count had been
  set just too low).  It's exceptional to have a zillion arguments
  and I don't think it's worth the complexity, especially since
  it turned out to have a serious performance hit.

* Remove quite a bit of ad-hoc cruft

* Move worthSplittingFun, worthSplittingThunk from WorkWrap to
  Demand. This allows JointDmd to be fully abstract, examined
  only inside Demand.

Everything else really follows from these changes.

All of this is really just refactoring, so we don't expect
big performance changes, but acutally the numbers look quite
good.  Here is a full nofib run with some highlights identified:

        Program           Size    Allocs   Runtime   Elapsed  TotalMem
--------------------------------------------------------------------------------
         expert          -2.6%    -15.5%      0.00      0.00     +0.0%
          fluid          -2.4%     -7.1%      0.01      0.01     +0.0%
             gg          -2.5%    -28.9%      0.02      0.02    -33.3%
      integrate          -2.6%     +3.2%     +2.6%     +2.6%     +0.0%
        mandel2          -2.6%     +4.2%      0.01      0.01     +0.0%
       nucleic2          -2.0%    -16.3%      0.11      0.11     +0.0%
           para          -2.6%    -20.0%    -11.8%    -11.7%     +0.0%
         parser          -2.5%    -17.9%      0.05      0.05     +0.0%
         prolog          -2.6%    -13.0%      0.00      0.00     +0.0%
         puzzle          -2.6%     +2.2%     +0.8%     +0.8%     +0.0%
        sorting          -2.6%    -35.9%      0.00      0.00     +0.0%
       treejoin          -2.6%    -52.2%     -9.8%     -9.9%     +0.0%
--------------------------------------------------------------------------------
            Min          -2.7%    -52.2%    -11.8%    -11.7%    -33.3%
            Max          -1.8%     +4.2%    +10.5%    +10.5%     +7.7%
 Geometric Mean          -2.5%     -2.8%     -0.4%     -0.5%     -0.4%

Things to note

* Binary sizes are smaller. I don't know why, but it's good.

* Allocation is sometiemes a *lot* smaller. I believe that all the big numbers
  (I checked treejoin, gg, sorting) arise from one place, namely a function
  GHC.IO.Encoding.UTF8.utf8_decode, which is strict in two Buffers both of
  which have several arugments.  Not w/w'ing both arguments (which is what
  we did before) has a big effect.  So the big win in actually somewhat
  accidental, gained by removing the "unboxing strategy" code.

* A couple of benchmarks allocate slightly more.  This turns out
  to be due to reboxing (integrate).  But the biggest increase is
  mandel2, and *that* turned out also to be a somewhat accidental
  loss of CSE, and pointed the way to doing better CSE: see Trac
  #7596.

* Runtimes are never very reliable, but seem to improve very slightly.

All in all, a good piece of work.  Thank you Ilya!

This will make it possible to write PrelRules that produce an Integer
result without having Integer arguments.

CorePrep has its own eta reducer (for tiresome reasons) and it was
being sloppy about making sure it didn't change termination behaviour.
Thanks to Michal Palka for discovering this.

CorePrep has its own eta reducer (for tiresome reasons) and it was
being sloppy about making sure it didn't change termination behaviour.
Thanks to Michal Palka for discovering this.

Two changes here

* The main change here is to enhance the FloatIn pass so that it can
  float case-bindings inwards.  In particular the case bindings for
  array indexing.

* Also change the code in Simplify, to allow a case on array
  indexing (ie can_fail is true) to be discarded altogether if its
  results are unused.

Lots of new comments in PrimOp about can_fail and has_side_effects

Some refactoring to share the FloatBind data structure between
FloatIn and FloatOut