Fixes #767.

Signed-off-by: Edward Z. Yang <ezyang@cs.stanford.edu>

Signed-off-by: Edward Z. Yang <ezyang@cs.stanford.edu>

* Properly render both depends and exe-depends in components. (by @ezyang)
* Import ComponentDeps qualified to avoid clashes
* Make ComponentDeps.zip compatible with containers<0.5

This fixes #220: new-build now builds, installs and adds executables to
PATH automatically if they show up in 'build-tools'.  However, there is
still more that could be done: the new behavior only applies to a
specific list of 'build-tools' (alex, happy, etc) which Cabal recognizes
out of the box.  The plan is to introduce a new 'tool-depends' field to
allow dependencies on other executables as well.

Signed-off-by: Edward Z. Yang <ezyang@cs.stanford.edu>

As requested by Duncan, the majority of fields that originally
lived in ElaboratedPackage now are moved to ElaboratedConfiguredPackage
under the prefix 'elab'.  Some code has gotten simpler as a result.

Signed-off-by: Edward Z. Yang <ezyang@cs.stanford.edu>

Signed-off-by: Edward Z. Yang <ezyang@cs.stanford.edu>

A bit of a megapatch.  Here's what's in it:

* First, a few miscellaneous utility functions and reexports
  in Cabal.  I could have split these into a separate commit
  but I was too lazy to.

* Distribution.Client.Install got refactored:
  instead of using PackageFixedDeps, it uses IsUnit
  instead.  This is because we weren't using ComponentDeps
  in a nontrivial way; we just need some graph structure
  and IsNode (with UnitId keys) fulfills that. I also removed the
  invariant checking and error reporting because it was
  being annoying (we check the invariants already in
  SolverInstallPlan).

* Look at Distribution.Client.ProjectPlanning.Types.
  This contains the primary type change: ElaboratedConfiguredPackage
  is now EITHER a monolithic ElaboratedPackage, or a per-component
  ElaboratedComponent (it should get renamed but I didn't do that
  in this patch.)  These are what we're going to store in our
  plans: if a package we're building has a Setup script which supports
  per-component builds, we'll explode it into a component.  Otherwise
  we'll keep it as a package.  We'll see codepaths for both
  throughout.

* OK, so the expansion happens in ProjectPlanning, mostly in
  'elaborateAndExpandSolverPackage'.  You should review the
  package hash computation code closely.  When we can separate
  components, we compute a hash for each INDEPENDENTLY.  This
  is good: we get more sharing.

* We need to adjust the target resolution and pruning code
  in ProjectOrchestration and ProjectPlanning.  I did a dumb
  but easy idea: if a user mentions 'packagename' in a
  target name, I spray the PackageTarget on every
  possibly relevant IPID in buildTargets', and then pare
  it down later.

* And of course there's code in ProjectBuilding to actual
  do a configure and then build.

* We change the layout of build directories so that we can
  track each component separately.  While I was doing that,
  I also added compiler and platform information.

Custom doesn't work yet because I need to give them their own
separate component, and teach Cabal how to build them specially.

Signed-off-by: Edward Z. Yang <ezyang@cs.stanford.edu>

The previous approach I took, though correct, was quite
confusing.  If I refactor InstallPlan to operate on a
per-component basis, then we'll automatically get support
for convenience libraries, which will ultimately cleaner.
(While we won't be able to get rid of support for whole
package installs, it will be safe to assume packages
using convenience libraries also support one-shot
configure.)

I didn't revert the support in cabal install; I'm not
planning on componentizing it.

Signed-off-by: Edward Z. Yang <ezyang@cs.stanford.edu>

Signed-off-by: Edward Z. Yang <ezyang@cs.stanford.edu>

In both cases it's optional and only added when the log file is used.
This will be used later in reporting the build outcomes.

I think these names are now a bit more induitive / less confusing and a
bit more consistent.

We now have BuildOutcome(s) to mean either failure or a build result,
and BuildResult to mean the result of a non-failing build.

This commit adds a data structure, 'RetryLog', which is like a difference list
for the 'Progress' type, except that it only supports efficient appends at
failures. Since the solver continually appends logs and calls 'tryWith' while
exploring the search tree, it is important for those operations to be efficient.
Afterwards, the solver converts the 'RetryLog' back to a 'Progress' so that it
can be processed with pattern matching in Log.hs and Message.hs.

Previously, the solver filtered out redundant backjumping messages twice, once
in 'Log.logToProgress' and again in 'Message.showMessages. However,
'showMessages' relied on the backjumping messages to determine where to insert
messages about missing packages. This led to missing "unknown package" messages
(part of issue #3617).

This commit removes the filtering in 'logToProgress', because it was redundant.

These were used previously for the Installed and Failed package states,
but these states are now gone.

Importantly this now means that we can have a serialisable InstallPlan
without the failure types having to be serialisable. This means we can
use things like SomeException which is not serialisable. Since the
traversal is done separately, the result of the traversal contains the
failure values, but this result set does not have to be serialised.

Eliminate the local executeInstallPlan. The main change is that the new
execute returns BuildResults instead of an upated InstallPlan. This has
a few knock-on conseuqnces for the code that looks at the result of the
build execution.

Currently we don't actually do that much with the results in the
new-build code path (though we should) so there's less disruption than
one might imagine. The biggest change is in the integration tests which
do inspect the execution result to check things worked or didn't work as
expected.

The equivalent change for the old build code path will be more
disruptive since it does a lot of stuff with the execution results.

This is intended to replace a couple existing executeInstallPlan
implementations.

The tests check that execute visits packages in reverse topological
order, for both serial and parallel job control. There's also a check
that serial execute and executionOrder use exactly the same order.

This is a replacement for the existing linearizeInstallPlan utils.
It uses the new separate traversal approach.

The main thing this adds is infrastructure for generating random
InstallPlans (which is not totally trivial given their structure).

Removing 'topdown' as a resolver choice means that 'choose' is also
obsolete and so it is removed too.

This implements the flag `--allow-older` which is the analogous to
`--allow-newer` acting on lower bounds.

The resulting code is more verbose, but it is more backwards-compatible
and actually is simpler to understand in some cases (because
CLibName uniquely identifies the "public library"; no faffing
about with package names to figure it out.)

Signed-off-by: Edward Z. Yang <ezyang@cs.stanford.edu>

Signed-off-by: Edward Z. Yang <ezyang@cs.stanford.edu>

Lots of changes:

    - When possible, we use the container infrastructure (sudo: false)
      rather than Google Compute Engine infrastructure (sudo: required).
      Unfortunately, we can't use GCE for the Linux builds, where
      reduced RAM available hoses are GHC build.

    - Switched from using ./Setup and old-style cabal to new-build.
      There are numerous great benefits but the best is that
      .cabal/store can be cached on Travis, leading to huge speedups
      on the build.  Downside is we need to string-and-ceiling-wax
      support for test/haddock/etc.

    - I stopped bootstrapping on every build we do; instead there
      is a separate bootstrap build we do to make sure that that
      is working.  This also speeds up the basic builds since
      we are not building Cabal/cabal-install multiple times.

    - There are some hacks.  The big one is setting CABAL_BUILDDIR
      explicitly; this smooths over quite a few infelicities in
      the current new-build implementation.

Signed-off-by: Edward Z. Yang <ezyang@cs.stanford.edu>

Signed-off-by: Edward Z. Yang <ezyang@cs.stanford.edu>

Now we monomorphize SolverInstallPlan so that its data structures
are non-parametric, and delete functions that are not needed.
Specifically:

    - GenericPlanPackage -> SolverPlanPackage, lose the
      type parameters, lose the Processing/Installed/Failed
      constructors (this makes some partial functions total! Yay!)

    - GenericInstallPlan -> SolverInstallPlan

    - PlanProblem -> SolverPlanProblem

    - Deleted ready, processing, completed, failed,
      preexisting

Signed-off-by: Edward Z. Yang <ezyang@cs.stanford.edu>

Now we copy-paste the contents of InstallPlan into SolverInstallPlan,
thus giving it a separate set of types.  For now, we don't do anything
else, e.g., remove unnecessary functions or specialize.

We need a new function 'fromSolverInstallPlan' akin to
'mapPreservingGraph' which can take an InstallPlan from the
old solver install plan to the new one.

Signed-off-by: Edward Z. Yang <ezyang@cs.stanford.edu>

This change avoids holding the whole log in memory.

This also adds a not yet used `AllowOlder` newtype

This is a preparatory refactoring propsed in #3466 for supporting `--allow-older`