GitLab

This patch fixes a fairly long-standing bug (dating back to 2015) in
RdrName.bestImport, namely

   commit 9376249b
   Author: Simon Peyton Jones <simonpj@microsoft.com>
   Date:   Wed Oct 28 17:16:55 2015 +0000

   Fix unused-import stuff in a better way

In that patch got the sense of the comparison back to front, and
thereby failed to implement the unused-import rules described in
  Note [Choosing the best import declaration] in RdrName

This led to Trac #13064 and #15393

Fixing this bug revealed a bunch of unused imports in libraries;
the ones in the GHC repo are part of this commit.

The two important changes are

* Fix the bug in bestImport

* Modified the rules by adding (a) in
     Note [Choosing the best import declaration] in RdrName
  Reason: the previosu rules made Trac #5211 go bad again.  And
  the new rule (a) makes sense to me.

In unravalling this I also ended up doing a few other things

* Refactor RnNames.ImportDeclUsage to use a [GlobalRdrElt] for the
  things that are used, rather than [AvailInfo]. This is simpler
  and more direct.

* Rename greParentName to greParent_maybe, to follow GHC
  naming conventions

* Delete dead code RdrName.greUsedRdrName

Bumps a few submodules.

Reviewers: hvr, goldfire, bgamari, simonmar, jrtc27

Subscribers: rwbarton, carter

Differential Revision: https://phabricator.haskell.org/D5312

This switches the compiler/ component to get compiled with
-XNoImplicitPrelude and a `import GhcPrelude` is inserted in all
modules.

This is motivated by the upcoming "Prelude" re-export of
`Semigroup((<>))` which would cause lots of name clashes in every
modulewhich imports also `Outputable`

Reviewers: austin, goldfire, bgamari, alanz, simonmar

Reviewed By: bgamari

Subscribers: goldfire, rwbarton, thomie, mpickering, bgamari

Differential Revision: https://phabricator.haskell.org/D3989

This is another take on https://phabricator.haskell.org/D3844.

This patch removes then need for haddock to reimplement the calculation
of exported names from modules. Instead when renaming export lists ghc
annotates each IE with its exported names.

Haddocks current  export logic has caused lots of trouble in the past
(on the Github issue tracker):
  - https://github.com/haskell/haddock/issues/121
  - https://github.com/haskell/haddock/issues/174
  - https://github.com/haskell/haddock/issues/225
  - https://github.com/haskell/haddock/issues/344
  - https://github.com/haskell/haddock/issues/584
  - https://github.com/haskell/haddock/issues/591
  - https://github.com/haskell/haddock/issues/597

Updates haddock submodule.

Reviewers: austin, bgamari, ezyang

Reviewed By: bgamari, ezyang

Subscribers: rwbarton, thomie

Differential Revision: https://phabricator.haskell.org/D3864

Summary:
See https://ghc.haskell.org/trac/ghc/wiki/ImplementingTreesThatGrow

This commit prepares the ground for a full extensible AST, by replacing the type
parameter for the hsSyn data types with a set of indices into type families,

    data GhcPs -- ^ Index for GHC parser output
    data GhcRn -- ^ Index for GHC renamer output
    data GhcTc -- ^ Index for GHC typechecker output

These are now used instead of `RdrName`, `Name` and `Id`/`TcId`/`Var`

Where the original name type is required in a polymorphic context, this is
accessible via the IdP type family, defined as

    type family IdP p
    type instance IdP GhcPs = RdrName
    type instance IdP GhcRn = Name
    type instance IdP GhcTc = Id

These types are declared in the new 'hsSyn/HsExtension.hs' module.

To gain a better understanding of the extension mechanism, it has been applied
to `HsLit` only, also replacing the `SourceText` fields in them with extension
types.

To preserve extension generality, a type class is introduced to capture the
`SourceText` interface, which must be honoured by all of the extension points
which originally had a `SourceText`.  The class is defined as

    class HasSourceText a where
      -- Provide setters to mimic existing constructors
      noSourceText  :: a
      sourceText    :: String -> a

      setSourceText :: SourceText -> a
      getSourceText :: a -> SourceText

And the constraint is captured in `SourceTextX`, which is a constraint type
listing all the extension points that make use of the class.

Updating Haddock submodule to match.

Test Plan: ./validate

Reviewers: simonpj, shayan-najd, goldfire, austin, bgamari

Subscribers: rwbarton, thomie, mpickering

Differential Revision: https://phabricator.haskell.org/D3609

Summary:
RnEnv contains functions which convertn RdrNames into Names.

RnUnbound contains helper functions for reporting and creating
unbound variables.

RnFixity contains code which maintains the fixity environent
whilst renaming.

RnUtils contains the other stuff in RnEnv.

Reviewers: austin, goldfire, bgamari

Subscribers: goldfire, rwbarton, thomie, snowleopard

Differential Revision: https://phabricator.haskell.org/D3436

Signed-off-by: Edward Z. Yang <ezyang@cs.stanford.edu>

Summary:
I observed a bug where if I modified the module which implemented
an hsig in another package, GHC would not recompile the signature
in this situation.

The root cause was that we were conflating modules from user
imports, and "system" module dependencies (from signature
merging and instantiation.) So this patch handles them separately.
Signed-off-by: Edward Z. Yang <ezyang@cs.stanford.edu>

Test Plan: validate

Reviewers: simonpj, bgamari, austin

Subscribers: rwbarton, thomie, snowleopard

Differential Revision: https://phabricator.haskell.org/D3381

Summary:
This is required to make Haddock work correctly.

Comes with a Haddock submodule update for better rendering.
Signed-off-by: Edward Z. Yang <ezyang@cs.stanford.edu>

Test Plan: validate

Reviewers: bgamari, austin

Subscribers: rwbarton, thomie

Differential Revision: https://phabricator.haskell.org/D3335

This at long last realizes the ideas for type-indexed Typeable discussed in A
Reflection on Types (#11011). The general sketch of the project is described on
the Wiki (Typeable/BenGamari). The general idea is that we are adding a type
index to `TypeRep`,

    data TypeRep (a :: k)

This index allows the typechecker to reason about the type represented by the `TypeRep`.
This index representation mechanism is exposed as `Type.Reflection`, which also provides
a number of patterns for inspecting `TypeRep`s,

```lang=haskell
pattern TRFun :: forall k (fun :: k). ()
              => forall (r1 :: RuntimeRep) (r2 :: RuntimeRep)
                        (arg :: TYPE r1) (res :: TYPE r2).
                 (k ~ Type, fun ~~ (arg -> res))
              => TypeRep arg
              -> TypeRep res
              -> TypeRep fun

pattern TRApp :: forall k2 (t :: k2). ()
              => forall k1 (a :: k1 -> k2) (b :: k1). (t ~ a b)
              => TypeRep a -> TypeRep b -> TypeRep t

-- | Pattern match on a type constructor.
pattern TRCon :: forall k (a :: k). TyCon -> TypeRep a

-- | Pattern match on a type constructor including its instantiated kind
-- variables.
pattern TRCon' :: forall k (a :: k). TyCon -> [SomeTypeRep] -> TypeRep a
```

In addition, we give the user access to the kind of a `TypeRep` (#10343),

    typeRepKind :: TypeRep (a :: k) -> TypeRep k

Moreover, all of this plays nicely with 8.2's levity polymorphism, including the
newly levity polymorphic (->) type constructor.

Library changes
---------------

The primary change here is the introduction of a Type.Reflection module to base.
This module provides access to the new type-indexed TypeRep introduced in this
patch. We also continue to provide the unindexed Data.Typeable interface, which
is simply a type synonym for the existentially quantified SomeTypeRep,

    data SomeTypeRep where SomeTypeRep :: TypeRep a -> SomeTypeRep

Naturally, this change also touched Data.Dynamic, which can now export the
Dynamic data constructor. Moreover, I removed a blanket reexport of
Data.Typeable from Data.Dynamic (which itself doesn't even import Data.Typeable
now).

We also add a kind heterogeneous type equality type, (:~~:), to
Data.Type.Equality.

Implementation
--------------

The implementation strategy is described in Note [Grand plan for Typeable] in
TcTypeable. None of it was difficult, but it did exercise a number of parts of
the new levity polymorphism story which had not yet been exercised, which took
some sorting out.

The rough idea is that we augment the TyCon produced for each type constructor
with information about the constructor's kind (which we call a KindRep). This
allows us to reconstruct the monomorphic result kind of an particular
instantiation of a type constructor given its kind arguments.

Unfortunately all of this takes a fair amount of work to generate and send
through the compilation pipeline. In particular, the KindReps can unfortunately
get quite large. Moreover, the simplifier will float out various pieces of them,
resulting in numerous top-level bindings. Consequently we mark the KindRep
bindings as noinline, ensuring that the float-outs don't make it into the
interface file. This is important since there is generally little benefit to
inlining KindReps and they would otherwise strongly affect compiler performance.

Performance
-----------

Initially I was hoping to also clear up the remaining holes in Typeable's
coverage by adding support for both unboxed tuples (#12409) and unboxed sums
(#13276). While the former was fairly straightforward, the latter ended up being
quite difficult: while the implementation can support them easily, enabling this
support causes thousands of Typeable bindings to be emitted to the GHC.Types as
each arity-N sum tycon brings with it N promoted datacons, each of which has a
KindRep whose size which itself scales with N. Doing this was simply too
expensive to be practical; consequently I've disabled support for the time
being.

Even after disabling sums this change regresses compiler performance far more
than I would like. In particular there are several testcases in the testsuite
which consist mostly of types which regress by over 30% in compiler allocations.
These include (considering the "bytes allocated" metric),

 * T1969:  +10%
 * T10858: +23%
 * T3294:  +19%
 * T5631:  +41%
 * T6048:  +23%
 * T9675:  +20%
 * T9872a: +5.2%
 * T9872d: +12%
 * T9233:  +10%
 * T10370: +34%
 * T12425: +30%
 * T12234: +16%
 * 13035:  +17%
 * T4029:  +6.1%

I've spent quite some time chasing down the source of this regression and while
I was able to make som improvements, I think this approach of generating
Typeable bindings at time of type definition is doomed to give us unnecessarily
large compile-time overhead.

In the future I think we should consider moving some of all of the Typeable
binding generation logic back to the solver (where it was prior to
91c6b1f5). I've opened #13261 documenting this
proposal.

Summary:
A number of changes:

- Keep the TcGblEnv from typechecking the local signature
  around when we do merging.  In particular, we setup
  tcg_imports and tcg_rdr_env according to the local
  signature.  This improves our error output (for example,
  see bkpfail04) and also fixes a bug with reexporting
  modules in signatures (see bkpreex07)

- Fix a bug in thinning, where if we had signature A(module A),
  this previously would have *thinned out* all of the inherited
  signatures.  Now we treat every inherited signature as having
  come from an import like "import A", so a module A reexport
  will pick them up.

- Recompilation checking now keeps track of dependent source files
  of the source signature; previously we forgot to retain this
  info.

There's a manual update too.
Signed-off-by: Edward Z. Yang <ezyang@cs.stanford.edu>

Test Plan: validate

Reviewers: bgamari, austin

Subscribers: thomie

Differential Revision: https://phabricator.haskell.org/D3133

Summary:
Recompilation avoidance checks if -this-unit-id has changed by relying
on the "wanted module" check in readIface ("Something is amiss...").
Unfortunately, this check didn't check if the instantiation made
sense, which meant that if you changed the signatures of a Backpack
package, we'd still treat the old signatures as up-to-date.

The way I fixed this was by having findAndReadIface take in a 'Module'
representing the /actual/ module we were intending to lookup.  We
convert this into the 'Module' we expect to see in 'mi_module' and
now do a more elaborate check that will also verify that instantiations
make sense.

Along the way, I robustified the logging infrastructure for
recompilation checking, and folded wrongIfaceModErr (which
was dead code) into the error message.
Signed-off-by: Edward Z. Yang <ezyang@cs.stanford.edu>

Test Plan: validate

Reviewers: bgamari, austin

Subscribers: thomie, snowleopard

Differential Revision: https://phabricator.haskell.org/D3130

While the intent is good, they are actually too aggressive:
if a module reexports an identifier from a signature, we will
warn (even though we should not!)  We'll have to implement
this differently.
Signed-off-by: Edward Z. Yang <ezyang@cs.stanford.edu>

Signed-off-by: Edward Z. Yang <ezyang@cs.stanford.edu>

Signed-off-by: Edward Z. Yang <ezyang@cs.stanford.edu>

Test Plan: validate

Reviewers: bgamari, simonpj, austin

Subscribers: thomie

Differential Revision: https://phabricator.haskell.org/D3101

Prior to this, I hadn't thought about orphan handling at all.
This commit implements the semantics that if a signature
(transitively) imports an orphan instance, that instance
is considered in scope no matter what the implementing module
is.  (As it turns out, this is the semantics that falls out
when orphans are recorded transitively.)

This patch fixes a few bugs:

1. Put semantic modules in dep_orphs rather than identity
   modules.
2. Don't put the implementing module in dep_orphs when
   merging signatures (this is a silly bug that happened
   because we were reusing calculateAvails, which is
   designed for imports. It mostly works for signature
   merging, except this case.)
3. When renaming a signature, blast in the orphans of the
   implementing module inside Dependencies.
Signed-off-by: Edward Z. Yang <ezyang@cs.stanford.edu>

Test Plan: validate

Reviewers: bgamari, austin

Reviewed By: bgamari

Subscribers: thomie

Differential Revision: https://phabricator.haskell.org/D3095

This patch converts the 4 lasting static flags (read from the command
line and unsafely stored in immutable global variables) into dynamic
flags. Most use cases have been converted into reading them from a DynFlags.

In cases for which we don't have easy access to a DynFlags, we read from
'unsafeGlobalDynFlags' that is set at the beginning of each 'runGhc'.
It's not perfect (not thread-safe) but it is still better as we can
set/unset these 4 flags before each run when using GHC API.

Updates haddock submodule.

Rebased and finished by: bgamari

Test Plan: validate

Reviewers: goldfire, erikd, hvr, austin, simonmar, bgamari

Reviewed By: simonmar

Subscribers: thomie

Differential Revision: https://phabricator.haskell.org/D2839

GHC Trac Issues: #8440

Summary:
While thesing, I realized this part of the implementation
didn't make very much sense, so I started working on some
documentation updates to try to make things more explainable.

The new docs are organized around the idea of a
"never exported TyThing" (a non-implicit TyThing that never
occurs in the export list of a module).  I also removed
some outdated information that predated the change of
ModIface to store Names rather than OccNames.
Signed-off-by: Edward Z. Yang <ezyang@cs.stanford.edu>

Reviewers: simonpj, bgamari, austin

Subscribers: thomie

Differential Revision: https://phabricator.haskell.org/D2989

Forgot to handle these!  In they go, plus a test case.
Signed-off-by: Edward Z. Yang <ezyang@cs.stanford.edu>

Summary:
A few related problems:

- CoAxioms, like DFuns, are implicit and never exported,
  so we have to make sure we treat them the same way as
  DFuns: in RnModIface we need to rename references to
  them with rnIfaceImplicit and in mergeSignatures we need
  to NOT check them directly for compatibility (the
  test on the type family will do this check for us.)

- But actually, we weren't checking if the axioms WERE
  consistent.  This is because we were forwarding all
  embedded CoAxiom references in the type family TyThing
  to the merged version, but that reference was what
  checkBootDeclM was using as a comparison point.
  This is similar to a problem we saw with DFuns.

  To fix this, I refactored the handling of implicit entities in TcIface
  for Backpack.  See Note [The implicit TypeEnv] for the gory details.
  Instead of passing the TypeEnv around explicitly, we stuffed it in
  IfLclEnv.
Signed-off-by: Edward Z. Yang <ezyang@cs.stanford.edu>

Test Plan: validate

Reviewers: bgamari, simonpj, austin

Subscribers: thomie

Differential Revision: https://phabricator.haskell.org/D2928

Summary:
Two major bug-fixes:

    - Check that fixities match between hsig and implementation

    - Merge and preserve fixities when merging signatures
Signed-off-by: Edward Z. Yang <ezyang@cs.stanford.edu>

Test Plan: validate

Reviewers: bgamari, simonpj, austin

Subscribers: thomie

Differential Revision: https://phabricator.haskell.org/D2919

GHC Trac Issues: #13066

Summary:
This won't stop people from attempting to use this identifier
(since it is still always going to be in the export list), but
having an explicit reference to something people shouldn't
use is a smell, so warn about it.
Signed-off-by: Edward Z. Yang <ezyang@cs.stanford.edu>

Test Plan: validate

Reviewers: simonpj, austin, bgamari

Subscribers: thomie

Differential Revision: https://phabricator.haskell.org/D2907

Summary:
If you use an inherited signature from another package in your own code,
the only valid PVP bound you can specify for this package is an *exact*
version bound.  This is because the signature is used both covariantly
(it provides declarations for import) and contravariantly (it specifies
what is required).  However, this is a bit distressing if you want to
use a PVP-style bound that allows for upgrading a package.  So there is
a dichotomy:

    1. Any signatures that come from packages with exact bounds
    (this includes, in particular, signature packages, who are
    included solely to make declarations available), can be
    used without problem by modules, but

    2. Any signatures that come from packages that are version
    bounded (i.e., any package that also provides modules) must
    NOT be used, because if they were used, they could break
    under a PVP policy that allows relaxations in the needed
    requirements.

To help users avoid situation (2), I've added a warning to all
signature declarations that come solely from (2).  This is not
perfect; you might still end up relying on some type identity
specified by a signature in a version-bounded package, but it
should help catch major errors.
Signed-off-by: Edward Z. Yang <ezyang@cs.stanford.edu>

Test Plan: validate

Reviewers: simonpj, austin, bgamari

Subscribers: thomie

Differential Revision: https://phabricator.haskell.org/D2906

Summary:
Generally speaking, it's not possible to "hide" a requirement from a
package you include, because if there is some module relying on that
requirement, well, you can't just wish it out of existence.

However, some packages don't have any modules.  For these, we can
validly thin out requirements; indeed, this is very convenient if
someone has published a large signature package but you only want
some of the definitions.

This patchset tweaks the interpretation of export lists in
signatures: in particular, they no longer need to refer to
entities that are defined locally; they range over both the current
signature as well as any signatures that were inherited from
signature packages (defined by having zero exposed modules.)

In the process of doing this, I cleaned up a number of other
things:

* rnModIface and rnModExports now report errors that occurred
  during renaming and can propagate these to the TcM monad.
  This is important because in the current semantics, you can
  thin out a type which is referenced by a value you keep;
  in this situation, we need to error (to ensure that all
  types in signatures are rooted, so that we can determine
  their identities).

* I ended up introducing a new construct 'dependency signature;
  to bkp files, to make it easier to tell if we were depending
  on a signature package.  It's not difficult for Cabal to
  figure this out (I already have a patch for it.)
Signed-off-by: Edward Z. Yang <ezyang@cs.stanford.edu>

Test Plan: validate

Reviewers: simonpj, austin, bgamari

Subscribers: thomie, mpickering

Differential Revision: https://phabricator.haskell.org/D2904

GHC Trac Issues: #12994

Summary:
If we didn't load the orphans, we might conclude an instance
is not implemented when it is.  See test bkp42.
Signed-off-by: Edward Z. Yang <ezyang@cs.stanford.edu>

Test Plan: validate

Reviewers: simonpj, austin, bgamari

Subscribers: thomie

Differential Revision: https://phabricator.haskell.org/D2841

Summary:
Previously, we always qualified names, even if they were defined
in the modules we were matching.  Adding the exports of the implementing
module into the RdrEnv greatly reduces the amount of qualification
(although we still can't qualify things that the signature *imported*.)
Signed-off-by: Edward Z. Yang <ezyang@cs.stanford.edu>

Test Plan: validate

Reviewers: simonpj, austin, bgamari

Subscribers: thomie

Differential Revision: https://phabricator.haskell.org/D2803

Summary:
Previously, we updated tcg_imports after doing all of the
actual matching, which was fine for outputting the interface,
but not good enough for checking if all type classes were
implemented; we weren't treating orphans as visible (when
they needed to be.)

Fixes #12945.
Signed-off-by: Edward Z. Yang <ezyang@cs.stanford.edu>

Test Plan: validate

Reviewers: simonpj, austin, bgamari

Subscribers: thomie

Differential Revision: https://phabricator.haskell.org/D2802

GHC Trac Issues: #12945

Summary:
Previously, we tested for type synonym loops by doing
a syntactic test on the literal type synonym declarations.
However, in some cases, loops could go through hs-boot
files, leading to an infinite loop (#12042); a similar
situation can occur when signature merging.

This commit replaces the syntactic test with a test on
TyCon, simply by walking down all type synonyms until
we bottom out, or find we've looped back.  It's a lot
simpler.
Signed-off-by: Edward Z. Yang <ezyang@cs.stanford.edu>

Test Plan: validate

Reviewers: simonpj, austin, bgamari

Subscribers: goldfire, thomie

Differential Revision: https://phabricator.haskell.org/D2656

GHC Trac Issues: #12042

Summary:
Previously, we deleted if the heads matched, which meant that
we effectively were picking an arbitrary instance if there
were incompatible instances.  The new behavior makes more sense,
although without incoherent instances you are unlikely to
be able to do anything useful with the instances.
Signed-off-by: Edward Z. Yang <ezyang@cs.stanford.edu>

Test Plan: validate

Reviewers: austin, bgamari

Subscribers: thomie

Differential Revision: https://phabricator.haskell.org/D2596

It turns out that we don't really need to be able to
extract a ComponentId from UnitId, except in one case.
So compress UnitId into a single FastString.

The one case where we do need the ComponentId is when
we are compiling an instantiated version of a package;
we need the ComponentId to look up the indefinite
version of this package from the database.  So now we
just pass it in as an argument -this-component-id.

Also: ghc-pkg now no longer will unregister a package if
you register one with the same package name, if the
instantiations don't match.

Cabal submodule update which tracks the same data type
change.
Signed-off-by: Edward Z. Yang <ezyang@cs.stanford.edu>

Signed-off-by: Edward Z. Yang <ezyang@cs.stanford.edu>

Summary:
This patch implements Backpack for GHC.  It's a big patch but I've tried quite
hard to keep things, by-in-large, self-contained.

The user facing specification for Backpack can be found at:

    https://github.com/ezyang/ghc-proposals/blob/backpack/proposals/0000-backpack.rst

A guide to the implementation can be found at:

    https://github.com/ezyang/ghc-proposals/blob/backpack-impl/proposals/0000-backpack-impl.rst

Has a submodule update for Cabal, as well as a submodule update
for filepath to handle more strict checking of cabal-version.
Signed-off-by: Edward Z. Yang <ezyang@cs.stanford.edu>

Test Plan: validate

Reviewers: simonpj, austin, simonmar, bgamari, goldfire

Subscribers: thomie, mpickering

Differential Revision: https://phabricator.haskell.org/D1482