This project is mirrored from https://github.com/haskell/Cabal. Pull mirroring updated .
  1. 26 Jul, 2016 3 commits
    • Duncan Coutts's avatar
      Annotate exceptions in downloading packages · 49b31ed7
      Duncan Coutts authored
      Download errors are now put into the residual install plan, like other
      build errors.
      
      Fixes issue #3387
      49b31ed7
    • Duncan Coutts's avatar
      Refactor async download code · db785667
      Duncan Coutts authored
      Split things up a little so the generic async fetch can live with the
      other fetch utils. This also makes it easier to test.
      
      Change the exception handling so that any exception in fetching is
      propagated when collecting the fetch result.
      db785667
    • Duncan Coutts's avatar
      Convert new-build to use the common BuildSuccess/Failure types · cdacc518
      Duncan Coutts authored
      As a result of the previous InstallPlan refactoring, we can now use the
      non-serialisable BuildFailure type from D.C.Types which uses
      SomeException, where previously we had to use a copy of that type that
      used String for the errors.
      
      So now there's no longer any need to have a separate set of types for
      BuildResult, BuildResults, BuildSuccess or BuildFailure. There was a
      minor difference in the structure of the BuildSuccess, where in the new
      build code we need to be able to produce the InstalledPackageInfo at a
      different point from the rest of the info in the BuildSuccess. This can
      be kept local to the ProjecBuilding module, but accounts for the
      somewhat larger number of changes in that module.
      cdacc518
  2. 25 Jul, 2016 3 commits
    • Duncan Coutts's avatar
      Remove the now-unused InstallPlan type args for result and failure · 0bb80be2
      Duncan Coutts authored
      These were used previously for the Installed and Failed package states,
      but these states are now gone.
      
      Importantly this now means that we can have a serialisable InstallPlan
      without the failure types having to be serialisable. This means we can
      use things like SomeException which is not serialisable. Since the
      traversal is done separately, the result of the traversal contains the
      failure values, but this result set does not have to be serialised.
      0bb80be2
    • Duncan Coutts's avatar
      Remove the now unused PlanPackage states · 7cb0844b
      Duncan Coutts authored
      We now just have the initial states, PreExisting and Configured. The
      Processing, Installed and Failed states are gone. We now do traversals
      separately from the InstallPlan, so we no longer need these states.
      7cb0844b
    • Duncan Coutts's avatar
      Convert new-build to use InstallPlan.execute · 97cff5c0
      Duncan Coutts authored
      Eliminate the local executeInstallPlan. The main change is that the new
      execute returns BuildResults instead of an upated InstallPlan. This has
      a few knock-on conseuqnces for the code that looks at the result of the
      build execution.
      
      Currently we don't actually do that much with the results in the
      new-build code path (though we should) so there's less disruption than
      one might imagine. The biggest change is in the integration tests which
      do inspect the execution result to check things worked or didn't work as
      expected.
      
      The equivalent change for the old build code path will be more
      disruptive since it does a lot of stuff with the execution results.
      97cff5c0
  3. 11 Jul, 2016 1 commit
  4. 26 Jun, 2016 1 commit
    • Duncan Coutts's avatar
      Don't initialise package dbs concurrently · 5adaf585
      Duncan Coutts authored
      Instead initialise all the ones we'll need in advance of building any
      packages. The alternative would have been to use a lock, but there's no
      particular advantage in trying to delay initialisation and avoiding
      locks keeps things simpler.
      
      This should fix #3460
      
      Another similar issue was fixed by 1acc00f8 which serialised the
      registration of packages in the local inplace package db, avoiding
      lost updates (which could previously be observed in test case T3460).
      5adaf585
  5. 20 Jun, 2016 1 commit
    • Duncan Coutts's avatar
      Change install lock to just a registration lock · 1acc00f8
      Duncan Coutts authored
      Allowing parallel installation of files, just serialising the actual
      registration. Also previously only store package installs were
      serialised but we also need to serialise registration of packages in the
      local inplace package db.
      1acc00f8
  6. 03 Jun, 2016 4 commits
    • Duncan Coutts's avatar
      Remove the now-redundant build JobLimit · 05079dd0
      Duncan Coutts authored
      The parallel JobControl now has a built-in maximum job limit, so the
      separate build JobLimit is unnecessary. The separate fetch JobLimit
      is still useful.
      05079dd0
    • Duncan Coutts's avatar
      Add a --keep-going flag and plumb it through · 53e7da37
      Duncan Coutts authored
      For both install and new-build.
      53e7da37
    • Duncan Coutts's avatar
      executeInstallPlan: mode to stop as soon as possible after failure · bf72b4f0
      Duncan Coutts authored
      Add a keepGoing :: Bool arg to control whether to keep going after a
      build failure or whether to try and stop as soon as possible.
      
      This makes use of the new JobControl interface to try cancelling
      outstanding jobs and also for checking if there are jobs in progress.
      
      For now the call sites are hard coded to use keepGoing = False.
      bf72b4f0
    • Duncan Coutts's avatar
      Extend the JobControl interface with cancelJobs · 9f3dd88c
      Duncan Coutts authored
      The motivation here is to support an install mode where we stop as soon
      as possible after one package fails, rather than going on as far as
      possible with other packages.
      
      To properly support this we need to be able to cancel jobs that have
      been submitted but that have not yet been started. This is easy for the
      serial JobControl but requires re-implementing the paralle JobControl.
      
      In the previous impl we simply fork a thread immediately for each job
      (and use a JobLimit externally to limit parallelim). The problem with
      that is that all submitted jobs are actually executing, so there's no
      way to cancel jobs that have not started yet (there are none). Combined
      with the executeInstallPlan behaviour of exposing the maximum possible
      paralleism this means that there can be a lot of jobs in progress
      (though most waiting on the JobLimit), and none of them can be sanely
      cancelled.
      
      So in this impl we have a built-in maximum parallelism, so we're
      executing at most that many jobs at once, while all others are waiting
      to be executed. The ones that are waiting can be cancelled.
      
      Because it's unspecified how many jobs can be cancelled, users can no
      longer rely on counting the number of outstanding jobs so we add a new
      method to check if there are any jobs remaining.
      
      Subsequent patches will add tests and make use of these new features to
      stop the install process as soon as possible after a failure, with a
      new --keep-going flag to get the existing behaviour.
      9f3dd88c
  7. 30 May, 2016 2 commits
    • Duncan Coutts's avatar
      Improve internal error checking to avoid issues like #3428 · 3d1f5ba9
      Duncan Coutts authored
      Check that we do get the registration info we expect in a couple places,
      and add detail to the error message originally reported in #3428.
      
      Also build the integration tests with assertions on, which might have
      caught this error earlier (via the invariant for the install plan).
      3d1f5ba9
    • Duncan Coutts's avatar
      Fix issue 3428, update the install plan with the right info · ba919898
      Duncan Coutts authored
      When building packages we update the install plan with the completed
      packages and for libraries we include the InstalledPackageInfo. We
      now support packages that register multiple libraries but only one of
      them is the representative public library, and only that one is stashed
      in the install plan. Out of the list of library registraions we select
      the primary one by looking for the one with the expected unit it. As
      part of collecting the registration info we do some processing (to
      account for older Cabal and ghc versions) and the mistake was that
      while did that post-processing ok and registered the right libraries
      we ended up returing the un-processed registraion info and so then
      failed to find the primary lib, and thus failed to stash any library
      info in the install plan, causing internal errors later on.
      
      So this patch fixes it to return the same post-processed registration
      info as actually gets registered. Subsequent patches improve the
      internal error checking for this issue.
      ba919898
  8. 09 May, 2016 1 commit
    • Duncan Coutts's avatar
      Add better exception handling in new-build · 51da3c74
      Duncan Coutts authored
      When executing the install plan, the various failures are supposed to be
      caught and put into the residual install plan. This adds the main cases
      for that, including the configure and build cases. Download is still not
      covered.
      
      This is a step towards better reporting of what failed when there are
      failures in deps etc.
      51da3c74
  9. 06 May, 2016 1 commit
  10. 05 May, 2016 1 commit
  11. 26 Apr, 2016 1 commit
  12. 10 Apr, 2016 2 commits
    • Edward Z. Yang's avatar
      Fix #3019, use sdist to find files to monitor. · 1f663390
      Edward Z. Yang authored
      
      
      As a refactoring, this moves allPackageSourceFiles from
      Distribution.Client.Sandbox.Timestamp to
      Distribution.Client.SrcDist, makes it thread safe, and has
      it return files relative to the source directory.
      Signed-off-by: default avatarEdward Z. Yang <ezyang@cs.stanford.edu>
      1f663390
    • Duncan Coutts's avatar
      Rename pkgConfigProgramDb/CompilerProgs to clarify purpose · a59f269b
      Duncan Coutts authored
      The ProgramDb in the ElaboratedSharedConfig actually only ever contains
      the programs configured by the compiler, and this is the way it will
      remain, so rename to clarify that usage.
      
      Config for other non-compiler programs will be handled differently. This
      makes sense because the compiler (and programs) have to be shared
      between all packages, while for non-compiler programs it's possible in
      principle to use different versions for different packages within a
      project. Thus it's not possible to have a single shared ProgramDb for
      all programs for all packages (other than for the compiler).
      a59f269b
  13. 05 Apr, 2016 1 commit
    • Edward Z. Yang's avatar
      Turn ReadyPackage into a newtype wrapper. · 137075dc
      Edward Z. Yang authored
      
      
      Previously, ReadyPackage was a ConfiguredPackage elaborated with
      a dependencies data structure which had InstalledPackageInfo
      rather than ConfiguredId. Well, it turned out that we only
      used the data from ConfiguredId! So that extra info is useless.
      
      Instead, ReadyPackage is now purely a newtype wrapper for type
      safety; a reminder that not all ConfiguredPackages can be built,
      only the ready ones.
      Signed-off-by: default avatarEdward Z. Yang <ezyang@cs.stanford.edu>
      137075dc
  14. 29 Mar, 2016 1 commit
  15. 27 Mar, 2016 2 commits
    • Duncan Coutts's avatar
      Extend file globs and file glob monitors · 2d569609
      Duncan Coutts authored and Mikhail Glushenkov's avatar Mikhail Glushenkov committed
      File globs can now be absolute, e.g. starting with / or c:\
      Also allow homedir relative, ie ~/
      
      Globs can also have a trailing slash, in which case they only match
      directories, not files.
      
      Previously whether globs could match dirs was not totally consistent.
      The matchFileGlob would match dirs, but the file monitor globs would
      not. The file monitor globs can now match dirs (or with a trailing
      slash, only match dirs). File monitors now also detect changes in the
      file type, ie switching from file to dir or the other way around.
      
      The file monitor are now pretty consistent between single file monitors
      and globs monitors. They now have equivalent capabilities and share
      code. For a single file or for a glob we can now control what we
      monitor if the path is a file or a dir. In both cases we can monitor
      mere existence, non-existence or modification time. For files we can
      additionally monitor content hash.
      
      File monitors now also detect changes in the file type, ie switching
      from file to dir or the other way around.
      
      New tests cover all these new file monitor cases. There are also new
      tests for glob syntax, covering printing/parsing round trips.
      
      (cherry picked from commit f6c1e71c)
      2d569609
    • Duncan Coutts's avatar
      Add new style project building · a6526403
      Duncan Coutts authored and Mikhail Glushenkov's avatar Mikhail Glushenkov committed
      This stage takes the ElaboratedInstallPlan and executes it. It does it
      in two passes.
      
      The first pass is the "dry run" pass where we work out which packages
      and components within packages we actually need to build. If we are in
      fact in --dry-run mode then of course this is the only pass we run, we
      have enough info after this to accurately report on what we would do.
      
      The first pass does the file monitor change stuff, which checks if
      package config has changed and probes for package file changes. Based on
      this we make a note of which phase of the build we would start at, up to
      and including skipping building that package entirely. In the latter
      case we replace the Configured package by a corresponding Installed
      package.
      
      The first pass also prunes the ElaboratedInstallPlan based on the
      targets that the user actually wants to build. For example we solve and
      plan on the basis that the user might want to run the test suites or
      benchmarks, but that doesn't mean we want to build them all the time
      (they'll usually pull in additional deps). So there's a moderately
      complex pass where we keep just the packages and components that are
      needed to build the target components.
      
      The first pass is moderately complicated but it makes no state changes
      and just returns the updated ElaboratedInstallPlan, so it should make
      things easier to debug.
      
      The second phase has very little logic it just does what its told based
      on all the earlier passes. For each package the dry run pass has worked
      out which build step to start with, e.g. downloading, configuring or
      building.
      
      The major cases the build phase has to deal with are packages that will
      be built and installed to the store, vs packages that are local and so
      are built inplace. It is only the latter that need all the complex
      rebuild checking. In both cases all the "setup $cmd" flags are
      calculated by the planning phase and we just do as we're told.
      
      (cherry picked from commit 300d9bad)
      a6526403