Convenience libraries are package-private libraries
that can be used as part of executables, libraries, etc
without being exposed to the external world.  Private
libraries are signified using the

    library foo

stanza.  Within a Cabal package, the name convenience library
shadows the conventional meaning of package name in
build-depends, so that references to "foo" do not indicate
foo in Hackage, but the convenience library defined in the
same package. (So, don't shadow Hackage packages!)

This commit implements convenience libraries such that they
ARE installed the package database (this prevents us from
having to special case dynamically linked executables);
in GHC 7.10 and later they are installed under the same
package name as the package that contained them, but have
a distinct "component ID" (one pay off of making the distinction
between component IDs and installed package IDs.)

There is a "default" library which is identified by the fact
that its library name coincides with the package name.  There
are some new convenience functions to permit referencing this.

There are a few latent bugs in this commit which are fixed
in later commits in this patchset.  (Those bugfixes required
a bit of refactoring, so it's clearer if they're not
with this patch.)

Signed-off-by: Edward Z. Yang <ezyang@cs.stanford.edu>

This custom setup ensures that if we are bootstrapping, we will
use the most recent version of Cabal.  (Without setup, cabal-install
will force usage of an old version of Cabal.)

Signed-off-by: Edward Z. Yang <ezyang@cs.stanford.edu>

Dependency solver quickcheck tests

Merge nix-local-build changes from 1.24

Duncan Coutts authored 8 years ago and Mikhail Glushenkov committed 8 years ago

And extend the project config print/parse tests to cover quoted strings.

(cherry picked from commit 72b93d91)

Duncan Coutts authored 8 years ago and Mikhail Glushenkov committed 8 years ago

File globs can now be absolute, e.g. starting with / or c:\
Also allow homedir relative, ie ~/

Globs can also have a trailing slash, in which case they only match
directories, not files.

Previously whether globs could match dirs was not totally consistent.
The matchFileGlob would match dirs, but the file monitor globs would
not. The file monitor globs can now match dirs (or with a trailing
slash, only match dirs). File monitors now also detect changes in the
file type, ie switching from file to dir or the other way around.

The file monitor are now pretty consistent between single file monitors
and globs monitors. They now have equivalent capabilities and share
code. For a single file or for a glob we can now control what we
monitor if the path is a file or a dir. In both cases we can monitor
mere existence, non-existence or modification time. For files we can
additionally monitor content hash.

File monitors now also detect changes in the file type, ie switching
from file to dir or the other way around.

New tests cover all these new file monitor cases. There are also new
tests for glob syntax, covering printing/parsing round trips.

(cherry picked from commit f6c1e71c)

Duncan Coutts authored 9 years ago and Mikhail Glushenkov committed 8 years ago

This stage takes the ElaboratedInstallPlan and executes it. It does it
in two passes.

The first pass is the "dry run" pass where we work out which packages
and components within packages we actually need to build. If we are in
fact in --dry-run mode then of course this is the only pass we run, we
have enough info after this to accurately report on what we would do.

The first pass does the file monitor change stuff, which checks if
package config has changed and probes for package file changes. Based on
this we make a note of which phase of the build we would start at, up to
and including skipping building that package entirely. In the latter
case we replace the Configured package by a corresponding Installed
package.

The first pass also prunes the ElaboratedInstallPlan based on the
targets that the user actually wants to build. For example we solve and
plan on the basis that the user might want to run the test suites or
benchmarks, but that doesn't mean we want to build them all the time
(they'll usually pull in additional deps). So there's a moderately
complex pass where we keep just the packages and components that are
needed to build the target components.

The first pass is moderately complicated but it makes no state changes
and just returns the updated ElaboratedInstallPlan, so it should make
things easier to debug.

The second phase has very little logic it just does what its told based
on all the earlier passes. For each package the dry run pass has worked
out which build step to start with, e.g. downloading, configuring or
building.

The major cases the build phase has to deal with are packages that will
be built and installed to the store, vs packages that are local and so
are built inplace. It is only the latter that need all the complex
rebuild checking. In both cases all the "setup $cmd" flags are
calculated by the planning phase and we just do as we're told.

(cherry picked from commit 300d9bad)

Duncan Coutts authored 9 years ago and Mikhail Glushenkov committed 8 years ago

There are two cases in which we replace source packages with ones that
are already installed: during planning when we improve the plan by
replacing configured source packages with pre-existing packages from the
store, but also there's just the case of skipping over a local package
that is already built and up to date. In the latter case we're replacing
a package in the Configured state not with one in the PreExisting state,
but with one in the Installed state, as if it had been built and
installed (which in a sense of course it has).

This new util is for the latter case.

(cherry picked from commit e869d803)

Duncan Coutts authored 9 years ago and Mikhail Glushenkov committed 8 years ago

Used in the next patch.

(cherry picked from commit 150c3189)

Duncan Coutts authored 9 years ago and Mikhail Glushenkov committed 8 years ago

This implements the planning phase of the new nix-style package build and
install system. In particular it includes the calculation of the
nix-style package ids by hashing all the package configuration.

Project planning is separated from project building.

The planning phase starts with finding the packages in the project and
then solving. We solve without looking at the installed packages in the
store. This makes everything more deterministic. The only installed
packages the solver looks at are the globally installed ones. This
approach also means we don't have any need of solver options like
--reinstall or --avoid-reinstalls etc.

The bulk of the planning phase is elaboration. We take the project
configuration and the solver's InstallPlan and elaborate the latter into
an ElaboratedInstallPlan. This is intended to contain all the details
needed for the build phase. For example all the "setup" command flags
are calculated directly from the elaborated plan.

The building phase is then intended to be much simpler and free of much
logic or policy. All of the complicated logic and policy is supposed to
be in the planning phase. This should also make things a lot easier to
debug, we can look at the plan we calculate and see if we're producing
the right build instructions, rather than debugging based on the actions
we end up executing.

Doing all the planning up front is also crucial to calculating nix-style
package hashes. This means we have the package ids up front. This then
allows us to have another up-front phase where we improve the plan by
replacing source packages with installed packages from the store.

All of this stuff is done in the Rebuild monad, with a few levels of
caches, so most of the time we can avoid recomputing the plan. In
particular we want to avoid re-running the solver unless we have to.

There are still quite a number of TODOs, which are categorised.

(cherry picked from commit 2d065c8c)

Require QuickCheck >= 2.8.2 for cabal-install unit tests

The 'ProjectConfig' tests use the Arbitrary instance for Map added in 2.8.2.

More detailed docs for the PACKAGETESTS flags.

Signed-off-by: Edward Z. Yang <ezyang@cs.stanford.edu>

Add current pkg info to generated `cabal_macros.h` file

Add the version string macro for the package currently being built
to the generated `dist/build/autogen/cabal_macros.h` file. Perviously
this file only contained the version strings of all the dependent
packages, but not the version string of the package itself.

Add the gen-bounds command.

Duncan Coutts authored 9 years ago and Mikhail Glushenkov committed 9 years ago

Two kinds of round-trip test:
 * type conversion ProjectConfig -> LegcyProjectConfig and back
 * ProjectConfig -> print -> parse
The latter goes out to the config file format and back.

These tests uncovered a number of issues in our general config code.

(cherry picked from commit e36c0e7e)

Duncan Coutts authored 9 years ago and Mikhail Glushenkov committed 9 years ago

We want to reuse the FlagAssignment printer/parser for the config files.

(cherry picked from commit d0fb249e)

Duncan Coutts authored 9 years ago and Mikhail Glushenkov committed 9 years ago

One for testing if globs are trivial, as in they're constant strings
without the possibility of matching more than one thing. This can help
with making better error messages.

Another util for matching file globs in the rebuild monad, and
automatically monitoring the glob.

(cherry picked from commit 4365bcb1)

Duncan Coutts authored 9 years ago and Mikhail Glushenkov committed 9 years ago

Getters and setters really need to match up.

Detected by parse/print round trip QC tests.

(cherry picked from commit a82af445)

Duncan Coutts authored 9 years ago and Mikhail Glushenkov committed 9 years ago

This defines the new cabal.project files and introduces the notion of a
project root (and the logic for finding it). Also has support for
implicit projects when no cabal.project file is defined.

Supports both reading and writing project files or fragments. The
printing & parsing round trips correctly. QC tests to follow.

This is a key part of the new nix-local-build branch approach, based
around projects with clear configuration state held in a project file
(or extra files).

This has support for file and dirs as packages within a project,
including by glob. It supports both globs that much match a target, and
optional globs that are allowed to match nothing. It has partial support
for local tarball, remote http tarball and remote source repo packages.

(cherry picked from commit 324b3240)

Duncan Coutts authored 9 years ago and Mikhail Glushenkov committed 9 years ago

This describes in one place the layout of the new dist dir, as used by
the nix-local-build branch. Also a similar approach to describing the
layout of the user-wide cabal directory.

The idea is that this centralises the description and makes it easier
to change and handle systematically (e.g. we have problems currently
with some user-wide files not being reolocatable).

(cherry picked from commit 7907a55c)

Duncan Coutts authored 9 years ago and Mikhail Glushenkov committed 9 years ago

The existing approach has been to parse top level fields and then
separately parse each top level section, and similarly for pretty
printing. A better approach is to follow the pattern for fields and have
a section description. Then we just parse or print fields+sections in
one call. And as a bonus, secitons can have subsections (and could even
do so recursively).

This patch just adds the new functionality. No existing config parsing
is switched over.

(cherry picked from commit 2f743b6d)

Duncan Coutts authored 9 years ago and Mikhail Glushenkov committed 9 years ago

withRepoContext keeps its current type (using GlobalFlags). Added
withRepoContext' that takes all the args seprately. Better name
suggestions welcome.

(cherry picked from commit 7b280cdd)

Fixes #3229.
(cherry picked from commit 7ab167b8)