A bit of a megapatch.  Here's what's in it:

* First, a few miscellaneous utility functions and reexports
  in Cabal.  I could have split these into a separate commit
  but I was too lazy to.

* Distribution.Client.Install got refactored:
  instead of using PackageFixedDeps, it uses IsUnit
  instead.  This is because we weren't using ComponentDeps
  in a nontrivial way; we just need some graph structure
  and IsNode (with UnitId keys) fulfills that. I also removed the
  invariant checking and error reporting because it was
  being annoying (we check the invariants already in
  SolverInstallPlan).

* Look at Distribution.Client.ProjectPlanning.Types.
  This contains the primary type change: ElaboratedConfiguredPackage
  is now EITHER a monolithic ElaboratedPackage, or a per-component
  ElaboratedComponent (it should get renamed but I didn't do that
  in this patch.)  These are what we're going to store in our
  plans: if a package we're building has a Setup script which supports
  per-component builds, we'll explode it into a component.  Otherwise
  we'll keep it as a package.  We'll see codepaths for both
  throughout.

* OK, so the expansion happens in ProjectPlanning, mostly in
  'elaborateAndExpandSolverPackage'.  You should review the
  package hash computation code closely.  When we can separate
  components, we compute a hash for each INDEPENDENTLY.  This
  is good: we get more sharing.

* We need to adjust the target resolution and pruning code
  in ProjectOrchestration and ProjectPlanning.  I did a dumb
  but easy idea: if a user mentions 'packagename' in a
  target name, I spray the PackageTarget on every
  possibly relevant IPID in buildTargets', and then pare
  it down later.

* And of course there's code in ProjectBuilding to actual
  do a configure and then build.

* We change the layout of build directories so that we can
  track each component separately.  While I was doing that,
  I also added compiler and platform information.

Custom doesn't work yet because I need to give them their own
separate component, and teach Cabal how to build them specially.

Signed-off-by: Edward Z. Yang <ezyang@cs.stanford.edu>

The previous approach I took, though correct, was quite
confusing.  If I refactor InstallPlan to operate on a
per-component basis, then we'll automatically get support
for convenience libraries, which will ultimately cleaner.
(While we won't be able to get rid of support for whole
package installs, it will be safe to assume packages
using convenience libraries also support one-shot
configure.)

I didn't revert the support in cabal install; I'm not
planning on componentizing it.

Signed-off-by: Edward Z. Yang <ezyang@cs.stanford.edu>

Described in: https://github.com/ghc-proposals/ghc-proposals/pull/4



./Setup configure now takes an argument to specify a specific
component name that should solely be configured.

Most of the gyrations in Configure are all about making it so that
we can feed in internal dependencies via --dependency.

I dropped the package name match sanity check to handle convenience
library package name munging.  Consider an internal library named
'q' in package 'p'.  When we install it to the package database,
we munged the package name into 'z-p-z-q', so that it doesn't
conflict with the actual package named 'q'.  Now consider when
we feed it in with --dependency q=p-0.1-hash-q.  Previously,
Cabal checked that the 'q' in --dependency matched the package
name in the database... which it doesn't. So I dropped the check.

I also had to make register/copy unconditionally install internal
libraries; otherwise you can't refer to them from later builds.

Also a miscellaneous refactor: convenience libraries are printed with a
"header" stanza now (not really a stanza header).

Signed-off-by: Edward Z. Yang <ezyang@cs.stanford.edu>

Signed-off-by: Edward Z. Yang <ezyang@cs.stanford.edu>

Signed-off-by: Edward Z. Yang <ezyang@cs.stanford.edu>

Signed-off-by: Edward Z. Yang <ezyang@cs.stanford.edu>

Signed-off-by: Edward Z. Yang <ezyang@cs.stanford.edu>

Make annotateFailure take a maybe log file, and add a
annotateFailureNoLog for the other case. This is a little more future
proof and simplifies things at the call sites.

Reporting build logs is important as we otherwise have no real info for
why deps failed to build. It does make the presentation more difficult
however because build logs can be long and in principle there can be
several, which means it can take up more than a single screen in a
console. The first thing users notice is typically the last few
messages, so in the case that we're presenting long build logs then its
important to also include a short summary at the end.

So our approach is this: for packages where we want to show a build log,
we dump those first, in full, each with a header to indicate which
package it is, log file name (for later reference), plus any extra
detail we have from the phase of the failure or the exception. Then
after all build logs we end with a short summary of the failure(s). For
packages where we do not show a build log (e.g. local packages that dump
live to the console) we only present a summary at the end, but we
include a little more detail than for the packages that had a build log
since this is the only thing we report for them. So we include details
of the exception.

The PlanningFailed case does not happen. The ReplFailed, HaddocksFailed
cases were previously covered by BuildFailed but they're worth
disinguishing.

In both cases it's optional and only added when the log file is used.
This will be used later in reporting the build outcomes.

We're going to alter and extend the BuildResult and BuildFailure types
for the new-build code, so we cannot share those types with the old
install code.

This patch doesn't change the structure, just redefines them locally and
switches uses to record style so we can add new fields easily.

I think these names are now a bit more induitive / less confusing and a
bit more consistent.

We now have BuildOutcome(s) to mean either failure or a build result,
and BuildResult to mean the result of a non-failing build.

[ci skip]

Signed-off-by: Edward Z. Yang <ezyang@cs.stanford.edu>

This commit adds a data structure, 'RetryLog', which is like a difference list
for the 'Progress' type, except that it only supports efficient appends at
failures. Since the solver continually appends logs and calls 'tryWith' while
exploring the search tree, it is important for those operations to be efficient.
Afterwards, the solver converts the 'RetryLog' back to a 'Progress' so that it
can be processed with pattern matching in Log.hs and Message.hs.

Signed-off-by: Edward Z. Yang <ezyang@cs.stanford.edu>

[ci skip]

This should fix issue #3394

Previous patches had made the changes to collect the detailed error info
all in one place. So this patch is just about deciding what to report
and how to report it.

Still TODO is mentioning log files, ie when the package build was being
logged to a file, we should include a snippet and say where the log file
can be found for full details.

lookup, direct deps and rev deps, rev dep closure.

If we were generating Haddock for cabal-install (we're not
currently) these would cause errors.  Make them stop causing
errors.

Signed-off-by: Edward Z. Yang <ezyang@cs.stanford.edu>

Fixes #3653

If 1.A is already linked to 0.A, then linking 2.A to 1.A gives the same result
as linking 2.A to 0.A. This commit avoids creating options to link to packages
that are already linked, to reduce the branching factor of the search tree.

Previously, the solver filtered out redundant backjumping messages twice, once
in 'Log.logToProgress' and again in 'Message.showMessages. However,
'showMessages' relied on the backjumping messages to determine where to insert
messages about missing packages. This led to missing "unknown package" messages
(part of issue #3617).

This commit removes the filtering in 'logToProgress', because it was redundant.

[ci skip]

Download errors are now put into the residual install plan, like other
build errors.

Fixes issue #3387

Split things up a little so the generic async fetch can live with the
other fetch utils. This also makes it easier to test.

Change the exception handling so that any exception in fetching is
propagated when collecting the fetch result.