Since we didn't really have a unit test setup for the solver yet, this
introduces some basic tests for solver, as well as tests for independent goals
specifically.

In particular, in the definition of dependencyInconsistencies.

One slightly annoying thing is that in order to validate an install plan, we
need to know if the goals are to be considered independent. This means we need
to pass an additional Bool to a few functions; to limit the number of functions
where this is necessary, also recorded whether or not goals are independent as
part of the InstallPlan itself.

This is implemented as a separate pass so that it can be understood
independently of the rest of the solver.

POption annotates a package choice with a "linked to" field. This commit
just introduces the datatype and deals with the immediate fallout, it doesn't
actually use the field for anything.

Duncan Coutts authored 10 years ago and Edsko de Vries committed 9 years ago

It turns out not to be the right solution for general private
dependencies and is just complicated. However we keep qualified
goals, just much simpler. Now dependencies simply inherit the
qualification of their parent goal. This gets us closer to the
intended behaviour for the --independent-goals feature, and for
the simpler case of private dependencies for setup scripts.

When not using --independent-goals, the solver behaves exactly as
before (tested by comparing solver logs for a hard hackage goal).
When using --independent-goals, now every dep of each independent
goal is qualified, so the dependencies are solved completely
independently (which is actually too much still).

Handle more 'strip --version' output formats.

Fixes #2497.

-Wall police

GHC HEAD now warns about redundant constraints.

Clean up graph traversal code

The option '-fno-missing-import-lists' is available in all supported GHC
versions (looks like it was introduced in 7.0).

Fixes #2469.

Minor type class cleanup

It's now only used in cabal-install.

[ci skip]

This addresses commit commemts

* https://github.com/haskell/cabal/pull/2488#commitcomment-10373353
* https://github.com/haskell/cabal/pull/2488#commitcomment-10373373
* https://github.com/haskell/cabal/pull/2488#commitcomment-10375768

from @23Skidoo.

See https://github.com/haskell-infra/hackage-trustees/issues/4#issuecomment-85492747

[ci skip]

Take advantage in cabal-install of the new
HasInstalledPackageId/PackagedInstall split in Cabal.  The graph traversal
functions in cabal-install, previously redundant, are now back in use. Their
types match the ones in Cabal, with only the difference in the PackageInstalled
(Cabal) versus PackageFixedDeps (cabal-install) type class.

The only PackageInstalled instance left in Cabal is for InstalledPackage, which
is a thin wrapper around InstalledPackageInfo; with these refactorings in
place, InstalledPackage is there only to support the TopDown solver. The fact
that we won't have PackageInstalled instances anymore for PlanPackage and co
means that we are forced to call the correct graph traversal functions (from
cabal-install, rather than from Cabal).

Introduce ConfiguredId:

    -- | A ConfiguredId is a package ID for a configured package.
    --
    -- Once we configure a source package we know it's InstalledPackageId (at
    -- least, in principle, even if we have to fake it currently). It is still
    -- however useful in lots of places to also know the source ID for the
    -- package.  We therefore bundle the two.
    --
    -- An already installed package of course is also "configured" (all it's
    -- configuration parameters and dependencies have been specified).
    --
    -- TODO: I wonder if it would make sense to promote this datatype to Cabal
    -- and use it consistently instead of InstalledPackageIds?
    data ConfiguredId = ConfiguredId {
        confSrcId  :: PackageId
      , confInstId :: InstalledPackageId
      }

And use it for ConfiguredPackage. As the comment says, though, I wonder if we
should use this in more places.

One slightly tricky thing here is that the output of both solvers had to be
modified to keep installed package IDs where possible; in the modular solver
this was easy enough, as it does this properly, but in the top-down solver this
is a bit of a hack; however, I’ve documented the hack in detail inline in the
code.

NOTE: Although this change is currently mostly cosmetic, in the future, once we
drop the single instance restriction, it is very important that we don't
convert from installed package IDs to source IDs and then back to installed
package IDs, as this conversion will be lossy.

Give the top-down solver it's own copy of `dependencyGraph`. This means that we
now have three independent implementations of `dependencyGraph`:

- `dependencyGraph` in `Cabal` takes a package index indexed by installed
  package IDs and only has access to library dependencies.
- `dependencyGraph` in `Distribution.Client.PlanIndex` in `cabal-install` takes
  a package index indexed by installed package IDs and has access to all
  dependencies.
- `dependencyGraph` in the top-down solver in `cabal-install` takes a package
  index indexed by package _names_, and has access to all dependencies.

Ideally we would switch the top-down solver over to use a package indexed by
installed package IDs, so that this duplication could be avoided, but that's a
bit of work and the top-down solver is legacy code anyway. Can still do that
later, of course.

Moreover, this makes the top-down solver monomorphic where possible, and
introduce its own SourceDeps class so that it is independent of the FixedDeps
class (which we will change over to use InstalledPackageIds instead).

Introduce

    dependencyClosure :: InstallPlan
                      -> [PackageIdentifier]
                      -> Either (PackageIndex PlanPackage) [(PlanPackage, [InstalledPackageId])]

And use this in the definition of `pruneInstallPlan` in `freeze`, to avoid
first converting an install plan from a `Cabal.PackageIndex` to a
`CabalInstall.PackageIndex`.

This resolves the first of the two irregularities mentioned in the previous
commit.

Both cabal-install and `Cabal` define a notion of a package index.
`Cabal` defines

    data PackageIndex a = PackageIndex !(Map InstalledPackageId a) !(Map PackageName (Map Version [a]))

whereas `cabal-install` defines

    newtype PackageIndex pkg = PackageIndex (Map PackageName [pkg])

Note that Cabal.PackageIndex is indexed by installed package IDs, whereas
CabalInstall.PackageIndex is indexed by package names.

There are a bunch of "graph traversal" functions that similarly duplicated
between `Cabal` and `cabal-install`; in `Cabal` we have

    brokenPackages            :: PackageInstalled a => PackageIndex a -> [(a, [InstalledPackageId])]
    dependencyClosure         :: PackageInstalled a => PackageIndex a -> [InstalledPackageId] -> Either (PackageIndex a) [(a, [InstalledPackageId])]
    dependencyCycles          :: PackageInstalled a => PackageIndex a -> [[a]]
    dependencyGraph           :: PackageInstalled a => PackageIndex a -> (Graph.Graph, Graph.Vertex -> a, InstalledPackageId -> Maybe Graph.Vertex)
    dependencyInconsistencies :: PackageInstalled a => PackageIndex a -> [(PackageName, [(PackageId, Version)])]
    reverseDependencyClosure  :: PackageInstalled a => PackageIndex a -> [InstalledPackageId] -> [a]
    reverseTopologicalOrder   :: PackageInstalled a => PackageIndex a -> [a]
    topologicalOrder          :: PackageInstalled a => PackageIndex a -> [a]

which are mirrored in `cabal-install` as

    brokenPackages            :: PackageFixedDeps pkg => PackageIndex pkg -> [(pkg, [PackageIdentifier])]
    dependencyClosure         :: PackageFixedDeps pkg => PackageIndex pkg -> [PackageIdentifier] -> Either (PackageIndex pkg) [(pkg, [PackageIdentifier])]
    dependencyCycles          :: PackageFixedDeps pkg => PackageIndex pkg -> [[pkg]]
    dependencyGraph           :: PackageFixedDeps pkg => PackageIndex pkg -> (Graph.Graph, Graph.Vertex -> pkg, PackageIdentifier -> Maybe Graph.Vertex)
    dependencyInconsistencies :: PackageFixedDeps pkg => PackageIndex pkg -> [(PackageName, [(PackageIdentifier, Version)])]
    reverseDependencyClosure  :: PackageFixedDeps pkg => PackageIndex pkg -> [PackageIdentifier] -> [pkg]
    reverseTopologicalOrder   :: PackageFixedDeps pkg => PackageIndex pkg -> [pkg]
    topologicalOrder          :: PackageFixedDeps pkg => PackageIndex pkg -> [pkg]

This by itself makes a certain amount of sense, but here's where the situation
gets confusing. `cabal-install` defines a `PlanIndex` as

    type PlanIndex = Cabal.PackageIndex PlanPackage

Note that is using `Cabal`'s notion of a PackageIndex, not `cabal-install`'s; it
makes sense that a PlanIndex is indexed by installed package IDs rather than
package names (even if currently we have to fake installed package IDs.

Almost all of the functions listed above, however, are only called on
`PlanIndex`s. This means that we invoke the functions from `Cabal`, not the
functions from `cabal-install`; in fact, almost all these functions in
`cabal-install` are completely unused right now.

    In `cabal-install`     but calls from `Cabal`
    ----------------------------------------------------------
    closed                 brokenPackages
    acyclic                dependencyCycles
    consistent             dependencyInconsistencies
    problems               brokenPackages', dependencyCycles',
                             dependencyInconsistencies'

This is more than just a code clean-up issue. As mentioned in the previous PR,
the fundamental difference between Cabal and cabal-install is their view of
dependencies: Cabal knows only about installed libraries and their library
dependencies, whereas cabal knows about packages and the dependencies of their
setup scripts, executables, test-suites, benchmarks, as well as their library
dependencies.

By calling the graph-traversal functions from `Cabal` rather than from
`cabal-install`, any of these additional dependencies are either completely
ignored, or else the distinction is lost (depending on how we implemented
installedDepends for plan packages); and neither option is correct.

For example, in `new` from Distribution.Client.InstallPlan (in `cabal-install`)
we call `dependendyGraph` on the plan index; since the plan index is defined in
terms of Cabal's plan index, we call Cabal's `dependencyGraph` here, but that
means that this graph will completely lack any setup dependencies. The reverse
graph is used in (only one place): `packagedThatDependOn`, which in turn is
(only) used in `failed`. But this is wrong: if a package fails to install, if
another package depends on it through a setup dependency, then that second
package should also be marked as impossible to install.

What needs to happen is that we modify the graph traversal functions from
`cabal-install` to take a PackageIndex from `Cabal` (so that we can apply them
to a PlanIndex), but use the dependencies from `FixedPackageDeps` rather than
the flat or incomplete dependencies we get from `PackageInstalled`. In fact,
the whole `PackageInstalled` instance for `ConfiguredPackage`, `ReadyPackage`
and `PlanPackage` should go: returning only part of the dependencies, or else
all dependencies flattened, is just too error prone.

This first commit only documents the problem (this commit message) and moves the
above functions to a new module called Distribution.Client.PlanIndex.

Cleaning this up is complicated by the fact that we _do_ still call two of the
above functions on a `CabalInstall.PackageIndex`:

* `pruneInstallPlan` from `Distribution.Client.Freeze` calls `dependencyClosure`
* The top-down solver calls `dependencyGraph`

If we change the above functions to work on a `Cabal.PackageIndex` instead these
two exceptions will break, so we need to look at that first.

Introduce a new superclass HasInstalledPackageId:

    class Package pkg => HasInstalledPackageId pkg where
      installedPackageId :: pkg -> InstalledPackageId

    class HasInstalledPackageId pkg => PackageInstalled pkg where
      installedDepends :: pkg -> [InstalledPackageId]

Most functions that deal with the package index now just require
HasInstalledPackageId; only the functions that actually require the
dependencies still rely on PackageInstalled.

The point is that a ConfiguredPackage/ReadyPackage/PlanPackage can reasonably
be made an instance of HasInstalledPackageId, but not of PackageInstalled; that
will be the topic of the next, much larger, pull request.

The fundamental difference between Cabal and cabal-install is that the former
deals with installed libraries, and -- in principle -- knows about _library_
dependencies only, whereas the latters deals with setup, executable, test-suite
and benchmark dependencies in addition to library dependencies. Currently we
classify all of these simply as 'dependencies' but that will change shortly.

In this commit we take a first step towards this by moving the PackageFixedDeps
class, which deals with dependencies of packages rather than installed
libraries, from Cabal to cabal-install.

The commit is pretty simple; we just move the type class and update import
statements where necessary.

Migrate the test suite to Tasty.

cabal-install.init: warn if the chosen package name is already registered to the source package index.

cabal-install.init: warn if the chosen package name is already registered to the source package index.