GitLab

Create a new syntax for depending on any library of any package.
The syntax is

    build-depends: pkgname:{pkgname, sublibname} -any

where the second `pkgname` specifies a dependency on the main unnamed
library.

Closes #4206.

This code existed to support building Cabal with 7.4 and earlier, but
we have recently dropped support for 7.4.

Fixes #4404.
Signed-off-by: Edward Z. Yang <ezyang@cs.stanford.edu>

The order of the tupling matters, because version needs to be
sorted lexicographically last in order to make the groupBy
work.  The old code was obviously wrong.
Signed-off-by: Edward Z. Yang <ezyang@cs.stanford.edu>

Signed-off-by: Edward Z. Yang <ezyang@cs.stanford.edu>

* Add missing record selector.

* Correctly mkPackageIndex

* Skip T4375 on GHC 8.2, it doesn't have a Cabal to actually use!
Signed-off-by: Edward Z. Yang <ezyang@cs.stanford.edu>

This makes the necessary changes to 4dc0f30fc36914ee2f01bde016bee98b8e0bb0bb
to handle component configuring correctly.  It probably is a good step
towards pkg:lib style dependencies.

The big ideas:

* There's a new AnnotatedId type, which is any identifier (like
  ComponentId), but also with a PackageId and ComponentName.
  It's a bit like ConfiguredId, but ConfiguredId is specialized
  for ComponentId.  There's a conversion function
  annotatedIdToConfiguredId.

* We adopt a totally new strategy for handling MungedPackageId in
  InstalledPackageInfo.  Rather than store the munged package
  identifier in IPI, we NEVER store it, instead computing it
  from the package name and library name (which are stored.)
  There is now code to PARSE a munged package name into these
  two components, so that when we read 'name' we get the
  correct info.  The resulting code is splendidly clear.

* Some places where we took a ComponentName (notably
  computeCompatPackageName) we now take 'Maybe UnqualComponentName'.
  This is more accurate, because compatibility package names are
  only computed for libraries, not other components.  Some code
  in Distribution.Backpack.ReadyComponent is partial now,
  but the invariants should hold up.

* A number of places where MungedId was applied, I undid them.
  Most notable are macro generation (that should be PACKAGES,
  not components) and mkLegacyUnitId (since REALLY old style
  unit identifiers, we won't support internal libraries anyway.)

* Many functions in PackageIndex are monomorphized to
  InstalledPackageInfo.  Fortunately cabal-install's usage
  still works.

* Distribution/Client/SolverPlanIndex.hs, not used by anyone,
  got the axe.

* Dependency solving has a hack to solve the problem of how to
  interpret installed internal libraries in the package database.
  The basic idea is that, although in most cases where we used
  a MungedId, we say it explicitly, in the SOLVER, we munge
  *installed package names* only, and keep the type PackageName.
  It's a hack, but the alternative is to write a lot more code.
  Note that is MORALLY PN is indeed a PackageName, since we really
  are solving for honest to goodness packages, not components!
  See Note [Index conversion with internal libraries] for more
  details.

* ConfiguredId now records the ComponentName.  This is quite important,
  because we need to use it to figure out how we are going to phrase
  --dependency.  In fact, it is a miracle that this worked at all
  (and it only worked because of a very skeevy update to the PackageId
  in the creation of ConfiguredComponent).  Irritatingly, this must
  be a Maybe ComponentName, because a ConfiguredId MIGHT refer to
  a setup component. Very distressing.
Signed-off-by: Edward Z. Yang <ezyang@cs.stanford.edu>

Signed-off-by: Edward Z. Yang <ezyang@cs.stanford.edu>

Previously, it would incorrectly report that we were
using incompatible versions of a package, if a package
showed up multiple times in Backpack.
Signed-off-by: Edward Z. Yang <ezyang@cs.stanford.edu>

This papers over the differences between Data.Map in containers
0.4.* and Data.Map.Strict in containers 0.5.*.

Signed-off-by: Edward Z. Yang <ezyang@cs.stanford.edu>

Also gets rid of `Distribution.Packages.TextClass`

Record the result of a slightly tricky debugging session so the next
person gets a head start.

[skip ci]

You never want those packages; you want the indefinite one
which you'll improve into a definite one.
Signed-off-by: Edward Z. Yang <ezyang@cs.stanford.edu>

Signed-off-by: Edward Z. Yang <ezyang@cs.stanford.edu>

New field, @instantiated-with@, which records the full
module substitution (it is dropped when we do 'improveUnitId').

For flexibility in the case of indefinite packages, some
occurences of Module are relaxed to IndefModule (exposedReexport).
This is just for convenience; in the case of a definite package
these reexports and instantiations are guaranteed to be 'Module's.

This patch also includes the minimal changes in other modules
needed due to the representation change.

'SimpleUnitId' constructor renamed to 'UnitId', and augmented
with a new field 'Maybe String' recording a hash that uniquely
identifies an instantiated unit of the library 'ComponentId'.
'UnitId' can't be used to represent partially instantiated
unit identifiers; see Distribution.Backpack for how we handle that.

Previous uses of 'SimpleUnitId' should now use 'newSimpleUnitId'.
'unitIdComponentId' folded into a record selector for 'ComponentId'.

When looking at heap-profiles of `cabal-install`, the `(:)` constructor
stands out as the most-allocated constructor on the heap.

Having to handle 10k+ package names contributes to the allocation
numbers, especially on 64bit archs where ASCII `String`s have a 24 byte
per character footprint.

This commit is a preparatory commit to pave the way for changing
`PackageName`'s internal representation to something like
`ShortByteString` (which is essentially a thin wrapper around primitive
`ByteArray#`s which themselves have have an overhead of 2 words + one
byte per ASCII character rounded up to nearest word) which would allow
to reduce the memory footprint by a full order of magnitude, as well as
reduce pointer chasing and GC overhead.

Signed-off-by: Edward Z. Yang <ezyang@cs.stanford.edu>

Two big ideas:

    * @--dependency@ takes a ComponentId, not UnitId.
      I used to think it should be a UnitId but it is
      now clear that you want to finger the indefinite
      unit id, which can be uniquely identified with
      a ComponentId

    * When hashing for an InstalledPackageId in
      new-build, we should produce a ComponentId,
      not a UnitId.

While cleaning up the results, for any codepaths which we don't plan on
implementing Backpack (Distribution.Client.Install, I'm looking at you),
just coerce ComponentId into UnitIds as necessary.
Signed-off-by: Edward Z. Yang <ezyang@cs.stanford.edu>

Signed-off-by: Edward Z. Yang <ezyang@cs.stanford.edu>

Signed-off-by: Edward Z. Yang <ezyang@cs.stanford.edu>

Signed-off-by: Edward Z. Yang <ezyang@cs.stanford.edu>

Fixes #3003.

Signed-off-by: Edward Z. Yang <ezyang@cs.stanford.edu>

GHC 8.0 is switching the state sponsored way to specify
linker names from -this-package-key to -this-unit-id, so
it behooves us to use the right one.  But it didn't make
much sense to pass ComponentIds to a flag named UnitId,
so I went ahead and finished a (planned) refactoring
to distinguish ComponentIds from UnitIds.

At the moment, there is NO difference between a ComponentId
and a UnitId; they are identical.  But semantically, a
component ID records what sources/flags we chose (giving us enough
information to typecheck a package), whereas a unit ID records
the component ID as well as how holes were instantiated
(giving us enough information to build it.)  MOST code
in the Cabal library wants unit IDs, but there are a few
places (macros and configuration) where we really do
want a component ID.

Some other refactorings that got caught up in here:

    - Changed the type of componentCompatPackageKey to String, reflecting the
      fact that it's not truly a UnitId or ComponentId.

    - Changed the behavior of CURRENT_PACKAGE_KEY to unconditionally
      give the compatibility package key, which is actually what you
      want if you're using it for the template Haskell trick.  I also
      added a CURRENT_COMPONENT_ID macro for the actual component ID,
      which is something that the Cabal test-suite will find useful.

    - Added the correct feature test for GHC 8.0 ("Uses unit IDs").
Signed-off-by: Edward Z. Yang <ezyang@cs.stanford.edu>

Signed-off-by: Edward Z. Yang <ezyang@cs.stanford.edu>

Instead of qualifying, in some cases I just added an extra
'hiding' pragma to squelch errors.  Common conflicts
(just grep for hiding):

    - Flag
        - Distribution.Simple.Compiler
        - Distribution.PackageDescription
        - Distribution.Simple.Setup
    - installedComponentId
        - Distribution.Package
        - Distribution.InstalledPackageInfo
    - doesFileExist
        - Distribution.PackageDescription.Check
    - instantiatedWith
        - I renamed the field in InstalledPackageInfo.  But
          it's probably going away with Backpack bits; I
          migth just excise it soon.
    - absoluteInstallDirs and substPathTemplate
        - Distribution.Simple.InstallDirs
        - Distribution.Simple.LocalBuildInfo

I fixed some shadowing errors by renaming local variables in some cases.
Common shadowings (we should perhaps consider not using these as
method/field names):

    - freeVars
    - components
    - noVersion
    - verbosity
    - get
    - description
    - name

Some data structures were moved around (IPIConvert and ABIHash)
to make it easier to handle import lists.

Some functions in Utils could be turned into reexports of standard
library functions.

No explicit imports were removed from non-Cabal imports.  These
imports help maintain warning cleanliness across versions of GHC,
so I don't recommend removing them.
Signed-off-by: Edward Z. Yang <ezyang@cs.stanford.edu>

This makes the code `-Wcompat`-clean with GHC 8.0

Due to the amount of `Monoid` instances, a compat-layer is used
rather than flooding the code-base with CPP conditionals.

Most types have these already. This just adds a few more.

This refactoring allows to drop `{-# LANGUAGE CPP #-}` from a couple of
modules again.

Today in Cabal, when you build and install a package, it is
uniquely identified using an InstalledPackageId which is computed
using the ABI hash of the library that was installed.  There
are few problems with doing it this way:

    - In a Nix-like world, we should instead uniquely identify
      build products by some sort of hash on the inputs to the
      compilation (source files, dependencies, flags).  The ABI
      hash doesn't capture any of this!

    - An InstalledPackageId suggests that we can uniquely identify
      build products by hashing the source and dependencies of
      a package as a whole.  But Cabal packages contain many components:
      a library, test suite, executables, etc.  Currently, when
      we say InstalledPackageId, we are really just talking about
      the dependencies of the library; however, this is unacceptable
      if a Cabal package can install multiple libraries; we need
      different identifiers for each.

    - We've also needed to compute another ID, which we've called
      the "package key", which is to be used for linker symbols
      and type equality GHC-side.  It is confusing what the distinction
      between this ID and InstalledPackageIds are; the main reason
      we needed another ID was because the package key was needed
      prior to compilation, whereas the ABI hash was only available
      afterwards.

This patch replaces InstalledPackageId and PackageKey with a
new identifier called ComponentId, which has the following
properties:

    - It is computed per-component, and consists of a package
      name, package version, hash of the ComponentIds
      of the dependencies it is built against, and the name
      of the component.  For example, "foo-0.1-abcdef" continues
      to identify the library of package foo-0.1, but
      "foo-0.1-123455-foo.exe" would identify the executable,
      and "foo-0.1-abcdef-bar" would identify a private sub-library
      named bar.

    - It is passed to GHC to be used for linker symbols and
      type equality.  So as far as GHC is concerned, this is
      the end-all be-all identifier.

    - Cabal the library has a simple, default routine for computing
      a ComponentId which DOES NOT hash source code;
      in a later patch Duncan is working on, cabal-install can
      specify a more detailed ComponentId for a package
      to be built with.

Here are some knock-on effects:

    - 'id' is a ComponentId

    - 'depends' is now a list of ComponentIds

    - New 'abi' field to record what the ABI of a unit is (as it is no longer
      computed by looking at the output of ghc --abi-hash).

    - The 'HasInstalledPackageId' typeclass is renamed to
      'HasComponentId'.

    - GHC 7.10 has explicit compatibility handling with
      a 'compatPackageKey' (an 'ComponentId') which is
      in a compatible format.  The value of this is read out
      from the 'key' field.
Signed-off-by: Edward Z. Yang <ezyang@cs.stanford.edu>

GHC HEAD now warns about redundant constraints.