This is (mostly) hvr's code.

It produces an external representation of the elaborated install plan.
The intention is for it to be used by external tools, editors etc, to
find out relevant details of the project and package configuration and
environment. It is updated whenever the elaborated install plan
changes, so it always reflects the current state.

Currently it only includes a modest subset of the details from the
elaborated install plan, but the goal is to include all relevant details
in a mostly-stable JSON format.

Currently it's based on the elaborated install plan prior to improving
the plan with packages from the store, but ultimately it will need to
include information from both before and after the 'improvement' phase.

You're not allowed to specify a different compiler program for different
packages within the same project. Given that in general we allow
specifying config about programs on a per-package basis this needs a
specific check.

Also add a place and a TODO for Future work in this area which is to
check up front that we can find the programs and to monitor them for
changes. Also, currently none of the program config ends up in the
package hashes, so it doesn't cause them to be rebuilt.

That is, the --with-$PROG program locations, the --$PROG-option(s)
program arguments and the --extra-prog-path search paths. These can be
passed on the command line or specified in the cabal.project file.

In the project file they can be specified for all local package or on a
per-package basis.

The compiler and its related programs are handled specially. Their
location cannot be specified independently for each package. A
subsequent patch will add the checks for this. Otherwise we just pass
the program config though.

Program options specified on the command line currently apply to all
local packages, not just the targets of the command. We may want to
revisit this.

Not needed immediately, but we'll need it eventually to avoid import
cycles later. No other changes, just moving things around and pruning
imports and language extensions.

The ProgramDb in the ElaboratedSharedConfig actually only ever contains
the programs configured by the compiler, and this is the way it will
remain, so rename to clarify that usage.

Config for other non-compiler programs will be handled differently. This
makes sense because the compiler (and programs) have to be shared
between all packages, while for non-compiler programs it's possible in
principle to use different versions for different packages within a
project. Thus it's not possible to have a single shared ProgramDb for
all programs for all packages (other than for the compiler).

Move the FlagAssignment from the project-wide all-packages config to the
per-package config.

Initially it had been easier to do it globally since it gets used as a
solver setting rather than as the other per-package config items, but
obviously it is supposed to be per-package.

So the flags field in the config top-level now applies to all local
packages. And it can also be specified in package-specific sections.

We don't yet check that any of the flags specified actually make sense
for either the local packages or for the specific packages.

It's a Map PackageName PackageConfig (or LegacyPackageConfig) so the
correct monoid instance is to merge the PackageConfig values.

Compute 'UnitId' when we compute a 'ConfiguredPackage';
consequently, we can eliminate 'FakeMap' (finally!)
There is one hack remaining, which is that 'SolverInstallPlan'
gins up fake unit IDs so that it can be keyed on UnitIds.
But this data structure exists only very briefly before
being converted into an 'InstallPlan' or 'ElaboratedInstallPlan'.

Signed-off-by: Edward Z. Yang <ezyang@cs.stanford.edu>

Currently, dependency solving immediately produces an 'InstallPlan'
which is then consumed by cabal install, or elaborated into
an 'ElaboratedInstallPlan' for cabal new-build.  However, this
translation is awkward, because the dependency solver knows nothing
about 'UnitId's, yet an 'InstallPlan' must indexed by 'UnitId's.
So there's a bit of faffing around to generate a "fake" unit id
to satisfy the interface, and then eventually correct it to the
right one.

So this patch starts moving us in a better direction, by introducing
a 'SolverInstallPlan', 'SolverPackage' and 'SolverId', intended
to be generated by the solver.  Then 'configureInstallPlan' or
'elaborateInstallPlan' elaborate this representation into the
representation needed by the destination.

The next step will be to generate the 'UnitId' during
'configureInstallPlan', and then we can get rid of the fake map
(so only Solver data types generate a fake identity, which
is only temporary until we generate 'UnitId's.)

Signed-off-by: Edward Z. Yang <ezyang@cs.stanford.edu>

Previously, ReadyPackage was a ConfiguredPackage elaborated with
a dependencies data structure which had InstalledPackageInfo
rather than ConfiguredId. Well, it turned out that we only
used the data from ConfiguredId! So that extra info is useless.

Instead, ReadyPackage is now purely a newtype wrapper for type
safety; a reminder that not all ConfiguredPackages can be built,
only the ready ones.

Signed-off-by: Edward Z. Yang <ezyang@cs.stanford.edu>

Signed-off-by: Edward Z. Yang <ezyang@cs.stanford.edu>

This commit fails its tests, because dynamic executables
linked against internal libraries aren't handled correctly
yet.  I had to do more refactoring to handle this correctly,
so it's in a separate commit.

Some refactoring in this one for identifying public libraries
as opposed to internal ones.

Signed-off-by: Edward Z. Yang <ezyang@cs.stanford.edu>

Convenience libraries are package-private libraries
that can be used as part of executables, libraries, etc
without being exposed to the external world.  Private
libraries are signified using the

    library foo

stanza.  Within a Cabal package, the name convenience library
shadows the conventional meaning of package name in
build-depends, so that references to "foo" do not indicate
foo in Hackage, but the convenience library defined in the
same package. (So, don't shadow Hackage packages!)

This commit implements convenience libraries such that they
ARE installed the package database (this prevents us from
having to special case dynamically linked executables);
in GHC 7.10 and later they are installed under the same
package name as the package that contained them, but have
a distinct "component ID" (one pay off of making the distinction
between component IDs and installed package IDs.)

There is a "default" library which is identified by the fact
that its library name coincides with the package name.  There
are some new convenience functions to permit referencing this.

There are a few latent bugs in this commit which are fixed
in later commits in this patchset.  (Those bugfixes required
a bit of refactoring, so it's clearer if they're not
with this patch.)

Signed-off-by: Edward Z. Yang <ezyang@cs.stanford.edu>

Duncan Coutts authored 8 years ago and Mikhail Glushenkov committed 8 years ago

File globs can now be absolute, e.g. starting with / or c:\
Also allow homedir relative, ie ~/

Globs can also have a trailing slash, in which case they only match
directories, not files.

Previously whether globs could match dirs was not totally consistent.
The matchFileGlob would match dirs, but the file monitor globs would
not. The file monitor globs can now match dirs (or with a trailing
slash, only match dirs). File monitors now also detect changes in the
file type, ie switching from file to dir or the other way around.

The file monitor are now pretty consistent between single file monitors
and globs monitors. They now have equivalent capabilities and share
code. For a single file or for a glob we can now control what we
monitor if the path is a file or a dir. In both cases we can monitor
mere existence, non-existence or modification time. For files we can
additionally monitor content hash.

File monitors now also detect changes in the file type, ie switching
from file to dir or the other way around.

New tests cover all these new file monitor cases. There are also new
tests for glob syntax, covering printing/parsing round trips.

(cherry picked from commit f6c1e71c)

Duncan Coutts authored 9 years ago and Mikhail Glushenkov committed 8 years ago

This implements the planning phase of the new nix-style package build and
install system. In particular it includes the calculation of the
nix-style package ids by hashing all the package configuration.

Project planning is separated from project building.

The planning phase starts with finding the packages in the project and
then solving. We solve without looking at the installed packages in the
store. This makes everything more deterministic. The only installed
packages the solver looks at are the globally installed ones. This
approach also means we don't have any need of solver options like
--reinstall or --avoid-reinstalls etc.

The bulk of the planning phase is elaboration. We take the project
configuration and the solver's InstallPlan and elaborate the latter into
an ElaboratedInstallPlan. This is intended to contain all the details
needed for the build phase. For example all the "setup" command flags
are calculated directly from the elaborated plan.

The building phase is then intended to be much simpler and free of much
logic or policy. All of the complicated logic and policy is supposed to
be in the planning phase. This should also make things a lot easier to
debug, we can look at the plan we calculate and see if we're producing
the right build instructions, rather than debugging based on the actions
we end up executing.

Doing all the planning up front is also crucial to calculating nix-style
package hashes. This means we have the package ids up front. This then
allows us to have another up-front phase where we improve the plan by
replacing source packages with installed packages from the store.

All of this stuff is done in the Rebuild monad, with a few levels of
caches, so most of the time we can avoid recomputing the plan. In
particular we want to avoid re-running the solver unless we have to.

There are still quite a number of TODOs, which are categorised.

(cherry picked from commit 2d065c8c)