GitLab

A new module with utilities for managing the store. This includes a new
approach for store updates that allows for concurrent updates to the
store from different processes.

This relies on the new file locking code. We log a message if we wait
for a store file lock. This should be a rare occurrence in practice,
but would help debugging if some other zombie process was holding the
file lock.

Adjust things so that instead of creating the store package db during
planning, we instead create them at the beginning of the building phase
along with the other package dbs we create at that point. That is,
create it in a write operation rather than in what is morally a read
operation.

Originally we had to do it in the planning phase because we used to read
the compiler package db to see what was installed, whereas now we just
use the store dir listing.

In fact we were already creating some package dbs during the building
phase, just not all of them, so this cleans things up in that respect.

If the dir does not exist then the result is [] and it monitors the dir
for non-existance. Previously it would just fail in this case. The new
behaviour is more useful.

So we have a spec of the layout of the store, independent of the
instance of the store within ~/.cabal/store. This will make new store
handling code cleaner by not entangling it with other global ~/.cabal
things.

We may also in future want to allow global and per-user stores, and we
certainly want the ability to specify a store location outside of
~/.cabal (though this could be done with the existing code too).

This code is backported from base from ghc-8.2.

It will be used as part of the concurrent store updates. It may also be
used in future to work around the lack of file locking in ghc-pkg prior
to version 8.2.

Turns out we need this for the approach we want to take for
concurrent store updates.

Remove the variants reregister and registerMultiInstance and generalise
the main register variant to cover them all. Introduce a RegisterOptions
record for the variations.

Eliminate an unused form where we supply a file rather than an
InstalledPackageInfo value.

The motivation is so we can more easily add yet more variations shortly.

This is an API change.

Add PATH to curl to slightly speed up build.

More accurately determine monitors to monitor.

Signed-off-by: Edward Z. Yang <ezyang@cs.stanford.edu>

Signed-off-by: Edward Z. Yang <ezyang@cs.stanford.edu>

Previously, we used the solver dependencies to figure out
what to monitor, but this lead to a failure mode where we
would also monitor the monitor files associated with the
test suite, which we really shouldn't do.

This patch pushes the monitor file calculation later until
the build step, at which point we know exactly what our
dependencies are, and monitor those precisely.

Fixes #4405.
Signed-off-by: Edward Z. Yang <ezyang@cs.stanford.edu>

Signed-off-by: Edward Z. Yang <ezyang@cs.stanford.edu>

Resurrect MungedId patch, and a pile of bugfixes

Signed-off-by: Edward Z. Yang <ezyang@cs.stanford.edu>

Signed-off-by: Edward Z. Yang <ezyang@cs.stanford.edu>

Signed-off-by: Edward Z. Yang <ezyang@cs.stanford.edu>

Signed-off-by: Edward Z. Yang <ezyang@cs.stanford.edu>

* Propagate ComponentName to ModuleSource, allowing
  us to accurately specify both package and library name
  of references.  (Arguably, should just stuff a ComponentInclude
  in there.)
Signed-off-by: Edward Z. Yang <ezyang@cs.stanford.edu>

* Add missing record selector.

* Correctly mkPackageIndex

* Skip T4375 on GHC 8.2, it doesn't have a Cabal to actually use!
Signed-off-by: Edward Z. Yang <ezyang@cs.stanford.edu>

Previously, configCompilerEx was setting Haddock as known,
but not actually configuring it; thus, later when we
required haddockProgram, we configured the WRONG program.
By configuring known programs, we can fix this problem.
Signed-off-by: Edward Z. Yang <ezyang@cs.stanford.edu>

This makes the necessary changes to 4dc0f30fc36914ee2f01bde016bee98b8e0bb0bb
to handle component configuring correctly.  It probably is a good step
towards pkg:lib style dependencies.

The big ideas:

* There's a new AnnotatedId type, which is any identifier (like
  ComponentId), but also with a PackageId and ComponentName.
  It's a bit like ConfiguredId, but ConfiguredId is specialized
  for ComponentId.  There's a conversion function
  annotatedIdToConfiguredId.

* We adopt a totally new strategy for handling MungedPackageId in
  InstalledPackageInfo.  Rather than store the munged package
  identifier in IPI, we NEVER store it, instead computing it
  from the package name and library name (which are stored.)
  There is now code to PARSE a munged package name into these
  two components, so that when we read 'name' we get the
  correct info.  The resulting code is splendidly clear.

* Some places where we took a ComponentName (notably
  computeCompatPackageName) we now take 'Maybe UnqualComponentName'.
  This is more accurate, because compatibility package names are
  only computed for libraries, not other components.  Some code
  in Distribution.Backpack.ReadyComponent is partial now,
  but the invariants should hold up.

* A number of places where MungedId was applied, I undid them.
  Most notable are macro generation (that should be PACKAGES,
  not components) and mkLegacyUnitId (since REALLY old style
  unit identifiers, we won't support internal libraries anyway.)

* Many functions in PackageIndex are monomorphized to
  InstalledPackageInfo.  Fortunately cabal-install's usage
  still works.

* Distribution/Client/SolverPlanIndex.hs, not used by anyone,
  got the axe.

* Dependency solving has a hack to solve the problem of how to
  interpret installed internal libraries in the package database.
  The basic idea is that, although in most cases where we used
  a MungedId, we say it explicitly, in the SOLVER, we munge
  *installed package names* only, and keep the type PackageName.
  It's a hack, but the alternative is to write a lot more code.
  Note that is MORALLY PN is indeed a PackageName, since we really
  are solving for honest to goodness packages, not components!
  See Note [Index conversion with internal libraries] for more
  details.

* ConfiguredId now records the ComponentName.  This is quite important,
  because we need to use it to figure out how we are going to phrase
  --dependency.  In fact, it is a miracle that this worked at all
  (and it only worked because of a very skeevy update to the PackageId
  in the creation of ConfiguredComponent).  Irritatingly, this must
  be a Maybe ComponentName, because a ConfiguredId MIGHT refer to
  a setup component. Very distressing.
Signed-off-by: Edward Z. Yang <ezyang@cs.stanford.edu>

This reverts commit 4774fb6558974a721176f1fb48d8ce7d43119251.

Signed-off-by: Edward Z. Yang <ezyang@cs.stanford.edu>

Signed-off-by: Edward Z. Yang <ezyang@cs.stanford.edu>

Signed-off-by: Edward Z. Yang <ezyang@cs.stanford.edu>

This reverts commit 332d809c, reversing
changes made to 0c72bc88.

Signed-off-by: Edward Z. Yang <ezyang@cs.stanford.edu>

Use a more common type of conflict in the solver basic memory usage test.

The test previously caused the solver to backtrack because of a missing package.
Missing packages are handled as a special case in the solver, so the test didn't
exercise as much code as it could have.  This commit causes the solver to
backtrack with a version conflict, which is a more common type of failure.
Other tests continue to test the missing package case.

Don't put VQuiet in VFlags; that's reserved for user-toggles.

Since VQuiet is set programmatically, it's inappropriate
for it to be propagated to the command line.  But unfortunately,
if it was set, we were accidentally using the fancy flag format.
This patch moves VQuiet to its own field in Verbosity so we
don't get confused.

Fixes #4393.
Signed-off-by: Edward Z. Yang <ezyang@cs.stanford.edu>

Filter out internal deps when forming package ElaboratedConfiguredPackage.

Signed-off-by: Edward Z. Yang <ezyang@cs.stanford.edu>

Signed-off-by: Edward Z. Yang <ezyang@cs.stanford.edu>

Prior to this patch, we were unconditionally sticking all dependencies
in a "whole package' ElaboratedConfiguredPackage, which resulted in
cycles when there were internal dependencies on the library in question,
and broken deps with convenience libraries.  Happily, the fix is simple,
although I did have to add ConfiguredId to the "extra" dependency fields
to make it easier to tell if a dependency was external or internal.

I added an extra test to make sure that we give a good error message
if a package requires per-component builds (apparently, internal
libraries require per-component builds!)

Fixes #4388.
Signed-off-by: Edward Z. Yang <ezyang@cs.stanford.edu>

Signed-off-by: Edward Z. Yang <ezyang@cs.stanford.edu>

ProjectPlanning refactor