GitLab

Signed-off-by: Edward Z. Yang <ezyang@cs.stanford.edu>

Summary:
Module signatures, like hs-boot files, are Haskell modules which omit
value definitions and contain only signatures.  This patchset implements
one particular aspect of module signature, namely compiling them against
a concrete implementation.  It works like this: when we compile an hsig
file, we must be told (via the -sig-of flag) what module this signature
is implementing.  The signature is compiled into an interface file which
reexports precisely the entities mentioned in the signature file.  We also
verify that the interface is compatible with the implementation.

This feature is useful in a few situations:

    1. Like explicit import lists, signatures can be used to reduce
    sensitivity to upstream changes.  However, a signature can be defined
    once and then reused by many modules.

    2. Signatures can be used to quickly check if a new upstream version
    is compatible, by typechecking just the signatures and not the actual
    modules.

    3. A signature can be used to mediate separate modular development,
    where the signature is used as a placeholder for functionality which
    is loaded in later.  (This is only half useful at the moment, since
    typechecking against signatures without implementations is not implemented
    in this patchset.)

Unlike hs-boot files, hsig files impose no performance overhead.

This patchset punts on the type class instances (and type families) problem:
instances simply leak from the implementation to the signature.  You can
explicitly specify what instances you expect to have, and those will be checked,
but you may get more instances than you asked for.  Our eventual plan is
to allow hiding instances, but to consider all transitively reachable instances
when considering overlap and soundness.

ToDo: signature merging: when a module is provided by multiple signatures
for the same base implementation, we should not consider this ambiguous.

ToDo: at the moment, signatures do not constitute use-sites, so if you
write a signature for a deprecated function, you won't get a warning
when you compile the signature.

Future work: The ability to feed in shaping information so that we can take
advantage of more type equalities than might be immediately evident.
Signed-off-by: Edward Z. Yang <ezyang@cs.stanford.edu>

Test Plan: validate and new tests

Reviewers: simonpj, simonmar, hvr, austin

Subscribers: simonmar, relrod, ezyang, carter, goldfire

Differential Revision: https://phabricator.haskell.org/D130

GHC Trac Issues: #9252

Summary:
This includes pretty much all the changes needed to make `Applicative`
a superclass of `Monad` finally. There's mostly reshuffling in the
interests of avoid orphans and boot files, but luckily we can resolve
all of them, pretty much. The only catch was that
Alternative/MonadPlus also had to go into Prelude to avoid this.

As a result, we must update the hsc2hs and haddock submodules.
Signed-off-by: Austin Seipp <austin@well-typed.com>

Test Plan: Build things, they might not explode horribly.

Reviewers: hvr, simonmar

Subscribers: simonmar

Differential Revision: https://phabricator.haskell.org/D13

Some others addressed as part of other recent patches.

in the previous patches in this series

Also start using the new package db file format properly, by using the
ghc-specific section.

This is the main patch in the series for removing the compiler's dep
on the Cabal lib.

Summary:
This patch set adds support for extra syntax on -package and related
arguments which allow you to thin and rename modules from a package.
For example, this argument:

    -package "base (Data.Bool as Bam, Data.List)"

adds two more modules into scope, Bam and Data.List, without adding
any of base's other modules to scope.

These flags are additive: so, for example, saying:

    -hide-all-packages -package base -package "base (Data.Bool as Bam)"

will provide both the normal bindings for modules in base, as well as
the module Bam.

There is also a new debug flag -ddump-mod-map which prints the state
of the module mapping database.  H = hidden, E = exposed (so for
example EH says the module in question is exported, but in a hidden
package.)

Module suggestions have been minorly overhauled to work better with reexports:
if you have -package "base (Data.Bool as Bam)" and mispell Bam, GHC
will suggest "Did you mean Bam (defined via package flags to be
base:Data.Bool)"; and generally you will get more accurate information.
Also, fix a bug where we suggest the -package flag when we really need
the -package-key flag.

NB: The renaming afforded here does *not* affect what wired in
symbols GHC generates.  (But it does affect implicit prelude!)

ToDo: add 'hiding' functionality, to make it easier to support the alternative
prelude use-case.

ToDo: Cabal support
Signed-off-by: Edward Z. Yang <ezyang@cs.stanford.edu>

Test Plan: new tests and validate

Reviewers: simonpj, simonmar, hvr, austin

Subscribers: simonmar, relrod, ezyang, carter

Differential Revision: https://phabricator.haskell.org/D113

GHC Trac Issues: #9375

Instead of building a multiply indirected data structure and querying
it on every import, we now have two data structures moduleToPkgConf
and moduleToPkgConfAll.  moduleToPkgConf is a single-level UniqFM that
is intended to be used for most valid imports; however, it does not
contain any information useful for error reporting.  If an error is
occurred, we then query moduleToPkgConfAll, which contains a more
comprehensive view of the package database.  This field is lazily
initialized (so this means we're retaining the package database list,
but this should be fine because we're already maintaining the entries
of the list.)  Additionally, the full view doesn't keep track of a boolean
toggle for visibility/exposure anymore, but instead tracks the *provenance*
of how the module binding came to be (the ModuleOrigin data type).

Additionally, we move the logic for determining if a module is exposed
or not from Finder.lhs and put it in Packages.lhs; this information is
communicated via the LookupResult data type.  Unfortunately, we can't
directly return a FindResult, because this data type is defined in
HscTypes which depends on Packages.  This is going to change some more
in the near future when I add thinning/renaming to package flags; the
error messages will need to be more flexible.

I've also slightly changed the semantics of error messages for package
qualified imports.  Previously, if we didn't find any package qualified
imports, but there were hidden modules in a *different* package, the error
message would prefer mentioning those as opposed to providing suggestions.
Now, if a module is hidden but in the wrong package, we won't mention it;
instead, it will get mentioned with the other module suggestions.  I
was too lazy to write a test, but I can add one if people would like.

The module reexport bug was, package q reexported p:P as Conflict,
and package r reexported p:P2 as Conflict, this was *not* reported as
a conflict, because the old logic incorrectly decided that P and P2 were
the same module on account of being from the same package.  The logic here
has been corrected.

Contains haddock submodule update.
Signed-off-by: Edward Z. Yang <ezyang@cs.stanford.edu>

Summary: Signed-off-by: Edward Z. Yang <ezyang@cs.stanford.edu>

Test Plan: validate

Reviewers: simonpj, simonmar, hvr, austin

Subscribers: simonmar, relrod, ezyang, carter

Differential Revision: https://phabricator.haskell.org/D107

This patch set makes us no longer assume that a package key is a human
readable string, leaving Cabal free to "do whatever it wants" to allocate
keys; we'll look up the PackageId in the database to display to the user.
This also means we have a new level of qualifier decisions to make at the
package level, and rewriting some Safe Haskell error reporting code to DTRT.

Additionally, we adjust the build system to use a new ghc-cabal output
Make variable PACKAGE_KEY to determine library names and other things,
rather than concatenating PACKAGE/VERSION as before.

Adds a new `-this-package-key` flag to subsume the old, erroneously named
`-package-name` flag, and `-package-key` to select packages by package key.

RFC: The md5 hashes are pretty tough on the eye, as far as the file
system is concerned :(

ToDo: safePkg01 test had its output updated, but the fix is not really right:
the rest of the dependencies are truncated due to the fact the we're only
grepping a single line, but ghc-pkg is wrapping its output.

ToDo: In a later commit, update all submodules to stop using -package-name
and use -this-package-key.  For now, we don't do it to avoid submodule
explosion.
Signed-off-by: Edward Z. Yang <ezyang@cs.stanford.edu>

Test Plan: validate

Reviewers: simonpj, simonmar, hvr, austin

Subscribers: simonmar, relrod, carter

Differential Revision: https://phabricator.haskell.org/D80

The general approach is to add a new field to the package database,
reexported-modules, which considered by the module finder as possible
module declarations.  Unlike declaring stub module files, multiple
reexports of the same physical package at the same name do not
result in an ambiguous import.

Has submodule updates for Cabal and haddock.

NB: When a reexport renames a module, that renaming is *not* accessible
from inside the package.  This is not so much a deliberate design choice
as for implementation expediency (reexport resolution happens only when
a package is in the package database.)

TODO: Error handling when there are duplicate reexports/etc is not very
well tested.
Signed-off-by: Edward Z. Yang <ezyang@cs.stanford.edu>

Conflicts:
	compiler/main/HscTypes.lhs
	testsuite/.gitignore
	utils/haddock

Summary:
Previously, both Cabal and GHC defined the type PackageId, and we expected
them to be roughly equivalent (but represented differently).  This refactoring
separates these two notions.

A package ID is a user-visible identifier; it's the thing you write in a
Cabal file, e.g. containers-0.9.  The components of this ID are semantically
meaningful, and decompose into a package name and a package vrsion.

A package key is an opaque identifier used by GHC to generate linking symbols.
Presently, it just consists of a package name and a package version, but
pursuant to #9265 we are planning to extend it to record other information.
Within a single executable, it uniquely identifies a package.  It is *not* an
InstalledPackageId, as the choice of a package key affects the ABI of a package
(whereas an InstalledPackageId is computed after compilation.)  Cabal computes
a package key for the package and passes it to GHC using -package-name (now
*extremely* misnamed).

As an added bonus, we don't have to worry about shadowing anymore.

As a follow on, we should introduce -current-package-key having the same role as
-package-name, and deprecate the old flag.  This commit is just renaming.

The haddock submodule needed to be updated.
Signed-off-by: Edward Z. Yang <ezyang@cs.stanford.edu>

Test Plan: validate

Reviewers: simonpj, simonmar, hvr, austin

Subscribers: simonmar, relrod, carter

Differential Revision: https://phabricator.haskell.org/D79

Conflicts:
	compiler/main/HscTypes.lhs
	compiler/main/Packages.lhs
	utils/haddock

In some cases, the layout of the LANGUAGE/OPTIONS_GHC lines has been
reorganized, while following the convention, to

- place `{-# LANGUAGE #-}` pragmas at the top of the source file, before
  any `{-# OPTIONS_GHC #-}`-lines.

- Moreover, if the list of language extensions fit into a single
  `{-# LANGUAGE ... -#}`-line (shorter than 80 characters), keep it on one
  line. Otherwise split into `{-# LANGUAGE ... -#}`-lines for each
  individual language extension. In both cases, try to keep the
  enumeration alphabetically ordered.
  (The latter layout is preferable as it's more diff-friendly)

While at it, this also replaces obsolete `{-# OPTIONS ... #-}` pragma
occurences by `{-# OPTIONS_GHC ... #-}` pragmas.

Mostly d -> g (matching DynFlag -> GeneralFlag).
Also renamed if* to when*, matching the Haskell if/when names

Now GHC still generates the _stub.c files, but the object file is
automatically merged into the main .o file for a module.  This means
that build systems (including GHC's own) no longer need to worry about
looking for _stub.o files and including them when linking.

I had to do lots of refactoring in DriverPipeline to make this work;
now there's a monad to carry around all the information, and
everything is a lot tidier.

The _stub.c is now created as a temporary file and removed after
compilation (unless the -keep-tmp-files flag is on).

The finder was reporting a hidden package when it meant a hidden
module, and vice versa (looks like a typo).

I'd done all the work to add fuzzy-match suggestions, but they
weren't really being used!  Here's what you get now

   module Foo where
    import Data.Lst

Foo.hs:3:1:
    Failed to load interface for `Data.Lst'
    Perhaps you meant
      Data.List (from base)
      Data.List (needs flag -package haskell2010-1.0.0.0)
      Data.Int (needs flag -package haskell2010-1.0.0.0)
    Use -v to see a list of the files searched for.

..so that you get a more helpful message when
you mis-spell a module name in an 'import'.

Validates, but not fully tested.

Based on Max's patch in Trac #2442, but heavily refactored.

Flushing and uncaching a single module is not completely atomic since
both caches a cleared separately.  However, flushing is only done when
changing the working directory which shouldn't be done concurrently to
other threads.  Uncaching is only done in 'summariseModule' which
requires some more work to become thread-safe anyway.

Fixes #3168

In GHCi if you say 'import Foo' meaning to load a package module Foo,
and Foo.hs is found on the search path, then GHCi replies "module Foo
is not loaded", because it knows Foo refers to the source file rather
than the package module, and you haven't loaded that module with
:load.

This is consistent with the usual module-finding semantics.  However,
it isn't particularly useful.  And it leads to silly problems like not
being able to start GHCi when you happen to be sitting in
libraries/base, because GHCi thinks the Prelude hasn't been loaded.

So now I've made a slight change to the way that 'import M' works: if
M is loaded, then it refers to the loaded module, otherwise it looks
for a package module M.  This does what the reporter of #2682 wanted,
and since it turns an error condition into meaningful behaviour it
can't break anything.  

The only undesirable consequence is that 'import M' might refer to a
different M than ':load M'.  Hopefully that won't lead to confusion.

Also remove some unused cruft

If the module was found in multiple hidden packages, we list them all.

Could not find module `Data.Generics':
  it is a member of the hidden package `base-3.0.3.0'
  it is a member of the hidden package `syb'
  Use -v to see a list of the files searched for.

> import System.Process
Could not find module `System.Process':
  There are files missing in the process-1.0.1.1 package,
  try running 'ghc-pkg check'.
  Use -v to see a list of the files searched for.

Now you can say
  
  import "network" Network.Socket

and get Network.Socket from package "network", even if there are
multiple Network.Socket modules in scope from different packages
and/or the current package.

This is not really intended for general use, it's mainly so that we
can build backwards-compatible versions of packages, where we need to
be able to do

module GHC.Base (module New.GHC.Base) where
import "base" GHC.Base as New.GHC.Base

These fix these failures:
   break008(ghci)
   break009(ghci)
   break026(ghci)
   ghci.prog009(ghci)
   ghci025(ghci)
   print007(ghci)
   prog001(ghci)
   prog002(ghci)
   prog003(ghci)
at least some of which have this symptom:
    Exception: expectJust prune