an empty target is an empty string

Speed up equal files check

Try update-alternatives ld

Speed up Distribution.Compat.CopyFile.filesEqual by checking size of files before checking their contents

Solver: Add individual flags to conflict sets.

[ci skip]

Enhance `--allow-{newer,older}` syntax

This extends the capabilities of `--allow-{newer,older}` to allow for more
fine-grained scoping to control more precisely which packages and constraints
a relaxation is applied to. See updated documentation for more details.

Move Distribution.Client.Compat.Prelude to Distribution.Compat.Prelude

The idea here is to break the dependency that has crept in from the
"Solver" into the "Client". (If there's ever going to be a separate
solver then the dependency would become a big problem.)

Completely randomizing the goal order exposes more bugs in backjumping than
using --reorder-goals.  I only applied the change to one test in this commit,
because the randomization function slowed down some of the other tests
significantly.

A DependencyReason contains flags paired with values FlagTrue, FlagFalse, or
FlagBoth, where FlagBoth means that the dependency was introduced by either
value of the flag and was lifted out of a conditional.  In the previous commit,
when a dependency led to a conflict, all flags in the DependencyReason were
added to the error message and conflict set.  This commit avoids adding flags
with value FlagBoth to the conflict set, because they can have a negative effect
on goal order: After the solver encounters a conflict involving such a flag, it
starts to prefer the flag when choosing goals, due to the solver's conflict
counting feature.  The solver shouldn't prefer the flag goal because it didn't
actually need to choose the flag to see the dependency.  Flags with value
FlagBoth are still useful for error messages, though.

I measured the performance of the last two commits by comparing commit
fc3ef2a7 (master) with this commit.  I did all
testing with GHC 8.0.1 and
'index-state(hackage.haskell.org) = 2017-06-11T02:16:39Z'.  I ran each version
of cabal (cabal install --dry-run) on each package on Hackage and looked for
packages with a difference in run time of at least 10%.  Then I reran cabal on
those packages to narrow down the list further (The results were noisy).  Here
are the average run times from five runs with the remaining packages:

package                        master result   master time (seconds)   branch result   branch time (seconds)   speedup
concraft                       backjump limit   5.67                   backjump limit   4.89                    1.16
forml                          backjump limit  10.26                   backjump limit   8.85                    1.16
hack-handler-simpleserver      backjump limit   6.28                   no solution      1.96                    3.21
hack-middleware-clientsession  backjump limit  17.86                   no solution      1.95                    9.14
haskore                        backjump limit   5.21                   no solution      1.67                    3.12
haskore-synthesizer            no solution     23.14                   no solution      2.21                   10.49
language-gcl                   no solution      3.06                   no solution      2.31                    1.33
ms                             backjump limit  61.68                   no solution      7.95                    7.76
nerf                           solution         4.1                    backjump limit   5.87                    0.7
puzzle-draw                    solution        56.28                   solution         5.31                   10.6
react-haskell                  backjump limit  44.23                   backjump limit  14.46                    3.06
reflex-transformers            no solution      2.18                   no solution      1.9                     1.15
shpider                        backjump limit   7.44                   solution         1.88                    3.95
thorn                          backjump limit  11.11                   backjump limit   4.44                    2.5

Both versions of cabal hit the backjump limit in four of the cases, so they
don't really tell us which version performed better.  All but one of the others
are improvements.  I looked at a few of the logs, and, as far as I can tell,
most of the improvements are due to flags being used in conflict counting.  This
behavior is similar to #4147, which was reverted.  The log for 'nerf', the one
package that had worse performance, showed that the solver chose some flags
earlier, but I didn't see any other reason for the slowdown.  The solver found a
solution when I used '--max-backjumps -1'.

These differences in run time are caused by differences in goal order and/or
backjumping, but I also wanted to compare performance when the solver went
through similar steps on master and the branch.  I found two commands that led
to very similar -v3 output between the cabal versions.  Presumably, package
flags didn't cause many conflicts in these runs.

cabal install --dry-run --max-backjumps -1 gi-glib happstack
cabal install --dry-run yesod

I took the average of three runs.  There wasn't a big difference between master
and the branch.

packages             result        master time (seconds)   branch time (seconds)   master memory (MB)   branch memory (MB)
gi-glib, happstack   no solution   30.02                   30.70                   245                  245
yesod                solution       3.53                    3.55                   239                  239.3

This commit changes the way that the solver tracks the variables that introduce
dependencies, in order to fix some bugs that prevented the solver from tracking
individual flags.  I refactored Dep, the type that represents a build-depends or
build-tool-depends dependency, so that it stores the package, flag, and stanza
choices that introduced the dependency.  That information is stored in a field
of type DependencyReason.  The DependencyReason is available to any solver phase
that needs to create conflict sets, such as "build" or "validation".  Whenever
there is a conflict involving a dependency, the solver creates a conflict set
and a log message using the dependency's DependencyReason.  This means that the
solver only needs to calculate the dependency reasons once, in
IndexConversion.hs, rather than every time they are used, i.e., in Builder.hs
(for goal reasons), Validate.hs, and Linking.hs.

Another difference between this commit and master is the dependencies between
solver variables.  On master, each dependency or variable is introduced by
exactly one other variable.  For example, if package x has a flag y, which
controls a dependency on package z, then the three variables depend on each
other in a chain, x -> y -> z.  If z can't be satisfied, the solver backjumps
to its cause, y, and if flipping y doesn't help, it continues backjumping to y's
cause, which is x.  In contrast, this commit allows each dependency to be
introduced by a package and any number of flags and stanzas.  So z's
DependencyReason would contain both x and y.

Storing all flags that introduced a dependency allows the solver to correctly
calculate conflict sets when there are nested conditionals.  We no longer need
to combine each package's flags into a single conflict set variable.  This
commit removes the 'simplifyVar' function and adds flag variables directly to
conflict sets.  See issue #4391.

This commit makes another minor change.  In this commit and master, if a
dependency appears in the if and else blocks of a conditional, the solver lifts
it out of the conditional.  Previously, the solver behaved as if the flag did
not introduce the dependency.  This commit adds the flag variable to the
conflict set or error message if the dependency is involved in a conflict.  This
change doesn't affect correctness, but I think it improves the error messages,
since it gives the whole reason that each dependency was introduced.

Fixes #4566.

Signed-off-by: Edward Z. Yang <ezyang@cs.stanford.edu>

Solver: Refactor Validate.hs.

Refactor goal sorting in the solver DSL.

This just moves some functions and types into Validate.hs in preparation for
#4391.

Now DSL.exResolve takes a goal-ordering function, which is more flexible
than a list of sorted goals.

[ci skip]

This reverts commit 8b563d4c.

Herbert Valerio Riedel authored 7 years ago and Mikhail Glushenkov committed 7 years ago

GHC 8.2.1 appears to take significantly longer to compile the code
and pushes us over the Travis job time limit.

[ci skip]

Fix test suite wobbliness on GHC 8.2.

Here were the root causes:

- Some tests involving Custom setpu showed MORE output
  (UseLocalPackageForSetup) when run on GHC 8.2.  This is because GHC
  8.2 ships a recent enough version of Cabal to know how to emit
  markers, which means we have started picking up the output.  I hacked
  up these tests to not accept this output, but a more correct thing to
  do is figure out how to NOT request marking of a Setup script which is
  not the inplace install.  This was a little tricky so I bailed.

- GHC 8.2 no longer emits "It is a member of the hidden package".
  This broke CustomWithoutCabalDefaultMain.  Not sure if this
  is a GHC regression but it's pretty harmless.

- While I was at it, I fixed an inexhaustive pattern match in
  cabal-testsuite (though perhaps poorly; I couldn't figure out
  what the new constructor does.)

Signed-off-by: Edward Z. Yang <ezyang@cs.stanford.edu>

Make new-haddock --haddock-for-hackage generate doc tarball

fixes #4521

Add constraints and preferences to the solver QuickCheck tests, and add a new test.

[ci skip]

Workaround invalid .cabal files with Main modules in other-extensions