This project is mirrored from https://github.com/haskell/Cabal. Pull mirroring updated .
  1. 24 Mar, 2016 3 commits
    • Duncan Coutts's avatar
      InstallPlan util to replace a ready package with an installed one · e869d803
      Duncan Coutts authored
      There are two cases in which we replace source packages with ones that
      are already installed: during planning when we improve the plan by
      replacing configured source packages with pre-existing packages from the
      store, but also there's just the case of skipping over a local package
      that is already built and up to date. In the latter case we're replacing
      a package in the Configured state not with one in the PreExisting state,
      but with one in the Installed state, as if it had been built and
      installed (which in a sense of course it has).
      
      This new util is for the latter case.
      e869d803
    • Duncan Coutts's avatar
      Split out reusable lookupReadyPackage util in InstallPlan · 150c3189
      Duncan Coutts authored
      Used in the next patch.
      150c3189
    • Duncan Coutts's avatar
      Add new style project planning · 2d065c8c
      Duncan Coutts authored
      This implements the planning phase of the new nix-style package build and
      install system. In particular it includes the calculation of the
      nix-style package ids by hashing all the package configuration.
      
      Project planning is separated from project building.
      
      The planning phase starts with finding the packages in the project and
      then solving. We solve without looking at the installed packages in the
      store. This makes everything more deterministic. The only installed
      packages the solver looks at are the globally installed ones. This
      approach also means we don't have any need of solver options like
      --reinstall or --avoid-reinstalls etc.
      
      The bulk of the planning phase is elaboration. We take the project
      configuration and the solver's InstallPlan and elaborate the latter into
      an ElaboratedInstallPlan. This is intended to contain all the details
      needed for the build phase. For example all the "setup" command flags
      are calculated directly from the elaborated plan.
      
      The building phase is then intended to be much simpler and free of much
      logic or policy. All of the complicated logic and policy is supposed to
      be in the planning phase. This should also make things a lot easier to
      debug, we can look at the plan we calculate and see if we're producing
      the right build instructions, rather than debugging based on the actions
      we end up executing.
      
      Doing all the planning up front is also crucial to calculating nix-style
      package hashes. This means we have the package ids up front. This then
      allows us to have another up-front phase where we improve the plan by
      replacing source packages with installed packages from the store.
      
      All of this stuff is done in the Rebuild monad, with a few levels of
      caches, so most of the time we can avoid recomputing the plan. In
      particular we want to avoid re-running the solver unless we have to.
      
      There are still quite a number of TODOs, which are categorised.
      2d065c8c
  2. 28 Jan, 2016 1 commit
  3. 27 Jan, 2016 4 commits
    • Duncan Coutts's avatar
      Simplify the graph dependency graph construction · bf5ac985
      Duncan Coutts authored
      Make the basic dependencyGraph construction function give us just a
      Vertex -> UnitId mapping, rather than a full Vertex -> pkg. Then in
      InstallPlan we keep that Vertex -> UnitId mapping and add a helper
      function that does the additional lookup to give us a Vertex -> pkg
      mapping. It's clearer this way, especially in the case of lookup errors.
      bf5ac985
    • Duncan Coutts's avatar
      Add a Binary instance for InstallPlan · 015ba31b
      Duncan Coutts authored
      This is easier to understand now that the canonical data vs the cached
      graph is clarified. We only store the main data, the cached info is
      reconstructed on deserialisation.
      015ba31b
    • Duncan Coutts's avatar
      Minor refactoring in InstallPlan to highlight caching approach · a7ff70ad
      Duncan Coutts authored
      This is to clarify the lazy caching strategy we use for the fields for
      the Graph (forward and reverse deps, and associated mappings).
      Reorder the fields, and mark non-lazy fields as strict. Make
      construction go via a smart constructor that fills in the cached fields.
      a7ff70ad
    • Duncan Coutts's avatar
      Add Binary instances for many types · 46aa019e
      Duncan Coutts authored
      So we can use them in binary cache files.
      
      Also relax version constraints on binary to work with binary-0.5.*,
      which requires that we expose Distribution.Compat.Binary from Cabal.
      
      D.Compat.Binary provides the Gerics support that we need to be able to
      derive instances when using binary-0.5. It's useful to be able to use
      binary-0.5 since that's the version bundled with older ghc versions.
      46aa019e
  4. 16 Jan, 2016 1 commit
    • Edward Z. Yang's avatar
      Distinguish between component ID and unit ID. · ef41f44e
      Edward Z. Yang authored
      
      
      GHC 8.0 is switching the state sponsored way to specify
      linker names from -this-package-key to -this-unit-id, so
      it behooves us to use the right one.  But it didn't make
      much sense to pass ComponentIds to a flag named UnitId,
      so I went ahead and finished a (planned) refactoring
      to distinguish ComponentIds from UnitIds.
      
      At the moment, there is NO difference between a ComponentId
      and a UnitId; they are identical.  But semantically, a
      component ID records what sources/flags we chose (giving us enough
      information to typecheck a package), whereas a unit ID records
      the component ID as well as how holes were instantiated
      (giving us enough information to build it.)  MOST code
      in the Cabal library wants unit IDs, but there are a few
      places (macros and configuration) where we really do
      want a component ID.
      
      Some other refactorings that got caught up in here:
      
          - Changed the type of componentCompatPackageKey to String, reflecting the
            fact that it's not truly a UnitId or ComponentId.
      
          - Changed the behavior of CURRENT_PACKAGE_KEY to unconditionally
            give the compatibility package key, which is actually what you
            want if you're using it for the template Haskell trick.  I also
            added a CURRENT_COMPONENT_ID macro for the actual component ID,
            which is something that the Cabal test-suite will find useful.
      
          - Added the correct feature test for GHC 8.0 ("Uses unit IDs").
      Signed-off-by: default avatarEdward Z. Yang <ezyang@cs.stanford.edu>
      ef41f44e
  5. 19 Oct, 2015 3 commits
    • Duncan Coutts's avatar
      Change a few InstallPlan functions to use all dependencies, including setup deps · 01d863ef
      Duncan Coutts authored
      When setup deps were introduced "Add ComponentSetup to ComponentDeps", we
      kept all uses to be the same old deps, without changing anything by using
      those new setup deps. In several places in InstallPlan it makes sense to
      use all deps, including setup deps. In fact for InstallPlan we should always
      use all deps except in one place: in checking plan validity where we check
      for dependency inconsistencies. That's the one place where we look only at
      "normal" non-setup deps to check that we're using only one version of each
      package.
      
      So for example, with this patch the freeze command now includes setup deps
      in the frozen set. The other case that should change is that failures of
      deps of packages used only for setup scripts should now be handled properly
      because we will look at setup dep edges when looking for things that depend
      on a failed package.
      01d863ef
    • Duncan Coutts's avatar
      Add other utils related to dependencyClosure, reverse and topological · 179e1da0
      Duncan Coutts authored
      These are also easy to implement based on the Graph in the InstallPlan.
      
      Not currently used, but will be eventually.
      179e1da0
    • Duncan Coutts's avatar
      Simplify InstallPlan.dependencyClosure and its single user · ed100d65
      Duncan Coutts authored
      We can calculate this directly on the Graph inside the InstallPlan
      rather that using a more complicated algorithm on the PlanIndex.
      
      This patch is primarily motivated by further changes to simplify
      and remove dead code from the PlanIndex module.
      ed100d65
  6. 09 Oct, 2015 1 commit
    • Edward Z. Yang's avatar
      Implement ComponentId, replacing PackageKey and InstalledPackageId. · b083151f
      Edward Z. Yang authored
      
      
      Today in Cabal, when you build and install a package, it is
      uniquely identified using an InstalledPackageId which is computed
      using the ABI hash of the library that was installed.  There
      are few problems with doing it this way:
      
          - In a Nix-like world, we should instead uniquely identify
            build products by some sort of hash on the inputs to the
            compilation (source files, dependencies, flags).  The ABI
            hash doesn't capture any of this!
      
          - An InstalledPackageId suggests that we can uniquely identify
            build products by hashing the source and dependencies of
            a package as a whole.  But Cabal packages contain many components:
            a library, test suite, executables, etc.  Currently, when
            we say InstalledPackageId, we are really just talking about
            the dependencies of the library; however, this is unacceptable
            if a Cabal package can install multiple libraries; we need
            different identifiers for each.
      
          - We've also needed to compute another ID, which we've called
            the "package key", which is to be used for linker symbols
            and type equality GHC-side.  It is confusing what the distinction
            between this ID and InstalledPackageIds are; the main reason
            we needed another ID was because the package key was needed
            prior to compilation, whereas the ABI hash was only available
            afterwards.
      
      This patch replaces InstalledPackageId and PackageKey with a
      new identifier called ComponentId, which has the following
      properties:
      
          - It is computed per-component, and consists of a package
            name, package version, hash of the ComponentIds
            of the dependencies it is built against, and the name
            of the component.  For example, "foo-0.1-abcdef" continues
            to identify the library of package foo-0.1, but
            "foo-0.1-123455-foo.exe" would identify the executable,
            and "foo-0.1-abcdef-bar" would identify a private sub-library
            named bar.
      
          - It is passed to GHC to be used for linker symbols and
            type equality.  So as far as GHC is concerned, this is
            the end-all be-all identifier.
      
          - Cabal the library has a simple, default routine for computing
            a ComponentId which DOES NOT hash source code;
            in a later patch Duncan is working on, cabal-install can
            specify a more detailed ComponentId for a package
            to be built with.
      
      Here are some knock-on effects:
      
          - 'id' is a ComponentId
      
          - 'depends' is now a list of ComponentIds
      
          - New 'abi' field to record what the ABI of a unit is (as it is no longer
            computed by looking at the output of ghc --abi-hash).
      
          - The 'HasInstalledPackageId' typeclass is renamed to
            'HasComponentId'.
      
          - GHC 7.10 has explicit compatibility handling with
            a 'compatPackageKey' (an 'ComponentId') which is
            in a compatible format.  The value of this is read out
            from the 'key' field.
      Signed-off-by: default avatarEdward Z. Yang <ezyang@cs.stanford.edu>
      b083151f
  7. 30 Jul, 2015 9 commits
  8. 06 Apr, 2015 3 commits
  9. 31 Mar, 2015 1 commit
    • Edsko de Vries's avatar
      Keep fine-grained deps after solver · 87a79be9
      Edsko de Vries authored
      The crucial change in this commit is the change to PackageFixedDeps to return a
      ComponentDeps structure, rather than a flat list of dependencies, as long with
      corresponding changes in ConfiguredPackage and ReadyPackage to accomodate this.
      
      We don't actually take _advantage_ of these more fine-grained dependencies yet;
      any use of
      
          depends
      
      is now a use of
      
         CD.flatDeps . depends
      
      but we will :)
      
      Note that I have not updated the top-down solver, so in the output of the
      top-down solver we cheat and pretend that all dependencies are library
      dependencies.
      87a79be9
  10. 27 Mar, 2015 1 commit
    • Edsko de Vries's avatar
      Deal with independent goals in plan validation · d56e1d8a
      Edsko de Vries authored
      In particular, in the definition of dependencyInconsistencies.
      
      One slightly annoying thing is that in order to validate an install plan, we
      need to know if the goals are to be considered independent. This means we need
      to pass an additional Bool to a few functions; to limit the number of functions
      where this is necessary, also recorded whether or not goals are independent as
      part of the InstallPlan itself.
      d56e1d8a
  11. 26 Mar, 2015 1 commit
  12. 25 Mar, 2015 1 commit
  13. 21 Mar, 2015 5 commits
    • Edsko de Vries's avatar
      Change PackageFixedDeps to use installed IDs · ea22d11f
      Edsko de Vries authored
      Take advantage in cabal-install of the new
      HasInstalledPackageId/PackagedInstall split in Cabal.  The graph traversal
      functions in cabal-install, previously redundant, are now back in use. Their
      types match the ones in Cabal, with only the difference in the PackageInstalled
      (Cabal) versus PackageFixedDeps (cabal-install) type class.
      
      The only PackageInstalled instance left in Cabal is for InstalledPackage, which
      is a thin wrapper around InstalledPackageInfo; with these refactorings in
      place, InstalledPackage is there only to support the TopDown solver. The fact
      that we won't have PackageInstalled instances anymore for PlanPackage and co
      means that we are forced to call the correct graph traversal functions (from
      cabal-install, rather than from Cabal).
      ea22d11f
    • Edsko de Vries's avatar
      Avoid forgetting known installed package IDs · daecdef9
      Edsko de Vries authored
      Introduce ConfiguredId:
      
          -- | A ConfiguredId is a package ID for a configured package.
          --
          -- Once we configure a source package we know it's InstalledPackageId (at
          -- least, in principle, even if we have to fake it currently). It is still
          -- however useful in lots of places to also know the source ID for the
          -- package.  We therefore bundle the two.
          --
          -- An already installed package of course is also "configured" (all it's
          -- configuration parameters and dependencies have been specified).
          --
          -- TODO: I wonder if it would make sense to promote this datatype to Cabal
          -- and use it consistently instead of InstalledPackageIds?
          data ConfiguredId = ConfiguredId {
              confSrcId  :: PackageId
            , confInstId :: InstalledPackageId
            }
      
      And use it for ConfiguredPackage. As the comment says, though, I wonder if we
      should use this in more places.
      
      One slightly tricky thing here is that the output of both solvers had to be
      modified to keep installed package IDs where possible; in the modular solver
      this was easy enough, as it does this properly, but in the top-down solver this
      is a bit of a hack; however, I’ve documented the hack in detail inline in the
      code.
      
      NOTE: Although this change is currently mostly cosmetic, in the future, once we
      drop the single instance restriction, it is very important that we don't
      convert from installed package IDs to source IDs and then back to installed
      package IDs, as this conversion will be lossy.
      daecdef9
    • Edsko de Vries's avatar
      Avoid package index conversion · e866b5e8
      Edsko de Vries authored
      Introduce
      
          dependencyClosure :: InstallPlan
                            -> [PackageIdentifier]
                            -> Either (PackageIndex PlanPackage) [(PlanPackage, [InstalledPackageId])]
      
      And use this in the definition of `pruneInstallPlan` in `freeze`, to avoid
      first converting an install plan from a `Cabal.PackageIndex` to a
      `CabalInstall.PackageIndex`.
      
      This resolves the first of the two irregularities mentioned in the previous
      commit.
      e866b5e8
    • Edsko de Vries's avatar
      Split the PackageInstalled class · 2ea3dde1
      Edsko de Vries authored
      Introduce a new superclass HasInstalledPackageId:
      
          class Package pkg => HasInstalledPackageId pkg where
            installedPackageId :: pkg -> InstalledPackageId
      
          class HasInstalledPackageId pkg => PackageInstalled pkg where
            installedDepends :: pkg -> [InstalledPackageId]
      
      Most functions that deal with the package index now just require
      HasInstalledPackageId; only the functions that actually require the
      dependencies still rely on PackageInstalled.
      
      The point is that a ConfiguredPackage/ReadyPackage/PlanPackage can reasonably
      be made an instance of HasInstalledPackageId, but not of PackageInstalled; that
      will be the topic of the next, much larger, pull request.
      2ea3dde1
    • Edsko de Vries's avatar
      Move PackageFixedDeps from Cabal to cabal-install · 445ad90d
      Edsko de Vries authored
      The fundamental difference between Cabal and cabal-install is that the former
      deals with installed libraries, and -- in principle -- knows about _library_
      dependencies only, whereas the latters deals with setup, executable, test-suite
      and benchmark dependencies in addition to library dependencies. Currently we
      classify all of these simply as 'dependencies' but that will change shortly.
      
      In this commit we take a first step towards this by moving the PackageFixedDeps
      class, which deals with dependencies of packages rather than installed
      libraries, from Cabal to cabal-install.
      
      The commit is pretty simple; we just move the type class and update import
      statements where necessary.
      445ad90d
  14. 10 Dec, 2014 1 commit
    • Luite Stegeman's avatar
      use CompilerInfo rather than CompilerId for resolving flags and · 7d91b773
      Luite Stegeman authored
      path templates.
      
      CompilerInfo contains more information about the compiler than
      CompilerId, which just stores the flavour and version. In particular,
      CompilerInfo knows an AbiTag, which can be used to tell binary
      incompatible results from the same compiler apart, and detailed
      information about supported languages and extensions.
      
      Some fields in CompilerInfo may be left unknown (Nothing). This can
      be used in the future to allow partially resolving configurations
      based on supported languages or extensions.
      7d91b773
  15. 07 Nov, 2014 1 commit
  16. 03 Oct, 2014 2 commits
  17. 23 Sep, 2014 1 commit
  18. 22 Sep, 2014 1 commit
    • Edward Z. Yang's avatar
      Fix three bugs with fake-map implementation for PackageIndex. · f59bab10
      Edward Z. Yang authored
      
      
      1. When we union PackageIndexes together, prefer the later one.
         This idiom is used when we update the processing-state of
         packages in an InstallPlan.
      
      2. dependencyInconsistencies' was missing a number of indirections
         through the fakeMap, so in some cases we incorrectly concluded
         packages were not equal when they were.
      
      3. We need to initialize the fakeMap with any pre-installed packages,
         otherwise the invariant check for configured-packages will fail.
      Signed-off-by: default avatarEdward Z. Yang <ezyang@cs.stanford.edu>
      f59bab10