This project is mirrored from Pull mirroring updated .
  1. 19 Sep, 2016 3 commits
    • Duncan Coutts's avatar
      Distinguish PreExisting vs Installed in BuildStatus · f3fc6504
      Duncan Coutts authored
      Not a big deal, but should be useful later for more precise status
      reporting. For now just means the rebuild reasons can be more precise.
    • Duncan Coutts's avatar
      Start using new InstallPlan.Installed state · 8f870aa0
      Duncan Coutts authored
      Change improvement and --dry-run phases to use Installed state rather
      than the PreExisting state. This means that PreExisting is now only used
      for installed packages from the global db, and never for installed
      packages from the store.
    • Duncan Coutts's avatar
      Add an Installed state to InstallPlan packages · 492db5b5
      Duncan Coutts authored
      This patch just adds the state without yet using it. That'll follow in
      subsequent patches.
      So why add an Installed state? Didn't we just remove the Installed,
      Processing and Failed states? Those states were used when we followed
      the approach of updating the InstallPlan as a build progressed (whereas
      we now do traversals without altering the InstallPlan).
      The idea of adding an Installed state now is that we can more usefully
      represent the state of the plan when we "improve" the plan with packages
      from the store or when we update the plan having checked if inplace
      packages are up to date. Currently in these two places we replace
      Configured source packages with PreExisting packages. There's a couple
      problems with this. Firstly the PreExisting state only contains an
      InstalledPackageInfo which means we loose information compared to all
      the detail in the Configured source package. This is relevant for things
      like plan.json output or other features that want to know the status of
      a project. Secondly we have to fake things for executables since they
      are not properly represented by InstalledPackageInfo.
  2. 09 Sep, 2016 1 commit
  3. 06 Sep, 2016 1 commit
  4. 24 Aug, 2016 1 commit
  5. 23 Aug, 2016 1 commit
  6. 21 Aug, 2016 6 commits
    • Edward Z. Yang's avatar
    • Edward Z. Yang's avatar
      Rewrite ElaboratedConfiguredPackage. · c41ead76
      Edward Z. Yang authored
      As requested by Duncan, the majority of fields that originally
      lived in ElaboratedPackage now are moved to ElaboratedConfiguredPackage
      under the prefix 'elab'.  Some code has gotten simpler as a result.
      Signed-off-by: default avatarEdward Z. Yang <>
    • Edward Z. Yang's avatar
    • Edward Z. Yang's avatar
      Be more careful about ComponentId versus UnitId. · bd7e2310
      Edward Z. Yang authored
      Two big ideas:
          * @--dependency@ takes a ComponentId, not UnitId.
            I used to think it should be a UnitId but it is
            now clear that you want to finger the indefinite
            unit id, which can be uniquely identified with
            a ComponentId
          * When hashing for an InstalledPackageId in
            new-build, we should produce a ComponentId,
            not a UnitId.
      While cleaning up the results, for any codepaths which we don't plan on
      implementing Backpack (Distribution.Client.Install, I'm looking at you),
      just coerce ComponentId into UnitIds as necessary.
      Signed-off-by: default avatarEdward Z. Yang <>
    • Edward Z. Yang's avatar
      Per-component new-build support (no Custom support yet). · d9bf6788
      Edward Z. Yang authored
      A bit of a megapatch.  Here's what's in it:
      * First, a few miscellaneous utility functions and reexports
        in Cabal.  I could have split these into a separate commit
        but I was too lazy to.
      * Distribution.Client.Install got refactored:
        instead of using PackageFixedDeps, it uses IsUnit
        instead.  This is because we weren't using ComponentDeps
        in a nontrivial way; we just need some graph structure
        and IsNode (with UnitId keys) fulfills that. I also removed the
        invariant checking and error reporting because it was
        being annoying (we check the invariants already in
      * Look at Distribution.Client.ProjectPlanning.Types.
        This contains the primary type change: ElaboratedConfiguredPackage
        is now EITHER a monolithic ElaboratedPackage, or a per-component
        ElaboratedComponent (it should get renamed but I didn't do that
        in this patch.)  These are what we're going to store in our
        plans: if a package we're building has a Setup script which supports
        per-component builds, we'll explode it into a component.  Otherwise
        we'll keep it as a package.  We'll see codepaths for both
      * OK, so the expansion happens in ProjectPlanning, mostly in
        'elaborateAndExpandSolverPackage'.  You should review the
        package hash computation code closely.  When we can separate
        components, we compute a hash for each INDEPENDENTLY.  This
        is good: we get more sharing.
      * We need to adjust the target resolution and pruning code
        in ProjectOrchestration and ProjectPlanning.  I did a dumb
        but easy idea: if a user mentions 'packagename' in a
        target name, I spray the PackageTarget on every
        possibly relevant IPID in buildTargets', and then pare
        it down later.
      * And of course there's code in ProjectBuilding to actual
        do a configure and then build.
      * We change the layout of build directories so that we can
        track each component separately.  While I was doing that,
        I also added compiler and platform information.
      Custom doesn't work yet because I need to give them their own
      separate component, and teach Cabal how to build them specially.
      Signed-off-by: default avatarEdward Z. Yang <>
    • Edward Z. Yang's avatar
      Undo new-build support for convenience libraries. · e6b6167d
      Edward Z. Yang authored
      The previous approach I took, though correct, was quite
      confusing.  If I refactor InstallPlan to operate on a
      per-component basis, then we'll automatically get support
      for convenience libraries, which will ultimately cleaner.
      (While we won't be able to get rid of support for whole
      package installs, it will be safe to assume packages
      using convenience libraries also support one-shot
      I didn't revert the support in cabal install; I'm not
      planning on componentizing it.
      Signed-off-by: default avatarEdward Z. Yang <>
  7. 11 Aug, 2016 1 commit
  8. 10 Aug, 2016 4 commits
  9. 07 Aug, 2016 1 commit
  10. 26 Jul, 2016 4 commits
    • Duncan Coutts's avatar
    • Duncan Coutts's avatar
      Annotate exceptions in downloading packages · 49b31ed7
      Duncan Coutts authored
      Download errors are now put into the residual install plan, like other
      build errors.
      Fixes issue #3387
    • Duncan Coutts's avatar
      Refactor async download code · db785667
      Duncan Coutts authored
      Split things up a little so the generic async fetch can live with the
      other fetch utils. This also makes it easier to test.
      Change the exception handling so that any exception in fetching is
      propagated when collecting the fetch result.
    • Duncan Coutts's avatar
      Convert new-build to use the common BuildSuccess/Failure types · cdacc518
      Duncan Coutts authored
      As a result of the previous InstallPlan refactoring, we can now use the
      non-serialisable BuildFailure type from D.C.Types which uses
      SomeException, where previously we had to use a copy of that type that
      used String for the errors.
      So now there's no longer any need to have a separate set of types for
      BuildResult, BuildResults, BuildSuccess or BuildFailure. There was a
      minor difference in the structure of the BuildSuccess, where in the new
      build code we need to be able to produce the InstalledPackageInfo at a
      different point from the rest of the info in the BuildSuccess. This can
      be kept local to the ProjecBuilding module, but accounts for the
      somewhat larger number of changes in that module.
  11. 25 Jul, 2016 3 commits
    • Duncan Coutts's avatar
      Remove the now-unused InstallPlan type args for result and failure · 0bb80be2
      Duncan Coutts authored
      These were used previously for the Installed and Failed package states,
      but these states are now gone.
      Importantly this now means that we can have a serialisable InstallPlan
      without the failure types having to be serialisable. This means we can
      use things like SomeException which is not serialisable. Since the
      traversal is done separately, the result of the traversal contains the
      failure values, but this result set does not have to be serialised.
    • Duncan Coutts's avatar
      Remove the now unused PlanPackage states · 7cb0844b
      Duncan Coutts authored
      We now just have the initial states, PreExisting and Configured. The
      Processing, Installed and Failed states are gone. We now do traversals
      separately from the InstallPlan, so we no longer need these states.
    • Duncan Coutts's avatar
      Convert new-build to use InstallPlan.execute · 97cff5c0
      Duncan Coutts authored
      Eliminate the local executeInstallPlan. The main change is that the new
      execute returns BuildResults instead of an upated InstallPlan. This has
      a few knock-on conseuqnces for the code that looks at the result of the
      build execution.
      Currently we don't actually do that much with the results in the
      new-build code path (though we should) so there's less disruption than
      one might imagine. The biggest change is in the integration tests which
      do inspect the execution result to check things worked or didn't work as
      The equivalent change for the old build code path will be more
      disruptive since it does a lot of stuff with the execution results.
  12. 11 Jul, 2016 1 commit
  13. 26 Jun, 2016 1 commit
    • Duncan Coutts's avatar
      Don't initialise package dbs concurrently · 5adaf585
      Duncan Coutts authored
      Instead initialise all the ones we'll need in advance of building any
      packages. The alternative would have been to use a lock, but there's no
      particular advantage in trying to delay initialisation and avoiding
      locks keeps things simpler.
      This should fix #3460
      Another similar issue was fixed by 1acc00f8 which serialised the
      registration of packages in the local inplace package db, avoiding
      lost updates (which could previously be observed in test case T3460).
  14. 20 Jun, 2016 1 commit
    • Duncan Coutts's avatar
      Change install lock to just a registration lock · 1acc00f8
      Duncan Coutts authored
      Allowing parallel installation of files, just serialising the actual
      registration. Also previously only store package installs were
      serialised but we also need to serialise registration of packages in the
      local inplace package db.
  15. 03 Jun, 2016 4 commits
    • Duncan Coutts's avatar
      Remove the now-redundant build JobLimit · 05079dd0
      Duncan Coutts authored
      The parallel JobControl now has a built-in maximum job limit, so the
      separate build JobLimit is unnecessary. The separate fetch JobLimit
      is still useful.
    • Duncan Coutts's avatar
      Add a --keep-going flag and plumb it through · 53e7da37
      Duncan Coutts authored
      For both install and new-build.
    • Duncan Coutts's avatar
      executeInstallPlan: mode to stop as soon as possible after failure · bf72b4f0
      Duncan Coutts authored
      Add a keepGoing :: Bool arg to control whether to keep going after a
      build failure or whether to try and stop as soon as possible.
      This makes use of the new JobControl interface to try cancelling
      outstanding jobs and also for checking if there are jobs in progress.
      For now the call sites are hard coded to use keepGoing = False.
    • Duncan Coutts's avatar
      Extend the JobControl interface with cancelJobs · 9f3dd88c
      Duncan Coutts authored
      The motivation here is to support an install mode where we stop as soon
      as possible after one package fails, rather than going on as far as
      possible with other packages.
      To properly support this we need to be able to cancel jobs that have
      been submitted but that have not yet been started. This is easy for the
      serial JobControl but requires re-implementing the paralle JobControl.
      In the previous impl we simply fork a thread immediately for each job
      (and use a JobLimit externally to limit parallelim). The problem with
      that is that all submitted jobs are actually executing, so there's no
      way to cancel jobs that have not started yet (there are none). Combined
      with the executeInstallPlan behaviour of exposing the maximum possible
      paralleism this means that there can be a lot of jobs in progress
      (though most waiting on the JobLimit), and none of them can be sanely
      So in this impl we have a built-in maximum parallelism, so we're
      executing at most that many jobs at once, while all others are waiting
      to be executed. The ones that are waiting can be cancelled.
      Because it's unspecified how many jobs can be cancelled, users can no
      longer rely on counting the number of outstanding jobs so we add a new
      method to check if there are any jobs remaining.
      Subsequent patches will add tests and make use of these new features to
      stop the install process as soon as possible after a failure, with a
      new --keep-going flag to get the existing behaviour.
  16. 30 May, 2016 2 commits
    • Duncan Coutts's avatar
      Improve internal error checking to avoid issues like #3428 · 3d1f5ba9
      Duncan Coutts authored
      Check that we do get the registration info we expect in a couple places,
      and add detail to the error message originally reported in #3428.
      Also build the integration tests with assertions on, which might have
      caught this error earlier (via the invariant for the install plan).
    • Duncan Coutts's avatar
      Fix issue 3428, update the install plan with the right info · ba919898
      Duncan Coutts authored
      When building packages we update the install plan with the completed
      packages and for libraries we include the InstalledPackageInfo. We
      now support packages that register multiple libraries but only one of
      them is the representative public library, and only that one is stashed
      in the install plan. Out of the list of library registraions we select
      the primary one by looking for the one with the expected unit it. As
      part of collecting the registration info we do some processing (to
      account for older Cabal and ghc versions) and the mistake was that
      while did that post-processing ok and registered the right libraries
      we ended up returing the un-processed registraion info and so then
      failed to find the primary lib, and thus failed to stash any library
      info in the install plan, causing internal errors later on.
      So this patch fixes it to return the same post-processed registration
      info as actually gets registered. Subsequent patches improve the
      internal error checking for this issue.
  17. 09 May, 2016 1 commit
    • Duncan Coutts's avatar
      Add better exception handling in new-build · 51da3c74
      Duncan Coutts authored
      When executing the install plan, the various failures are supposed to be
      caught and put into the residual install plan. This adds the main cases
      for that, including the configure and build cases. Download is still not
      This is a step towards better reporting of what failed when there are
      failures in deps etc.
  18. 06 May, 2016 1 commit
  19. 05 May, 2016 1 commit
  20. 26 Apr, 2016 1 commit
  21. 10 Apr, 2016 1 commit