This project is mirrored from Pull mirroring updated .
  1. 29 Sep, 2016 3 commits
    • Duncan Coutts's avatar
      Add runProjectPostBuildPhase in project orchestration · 66cd8579
      Duncan Coutts authored
      Previously the post-build phase was just reportBuildFailures and was
      called directly from Cmd{Build,Repl}. So initially this just rearranges
      things without changing behaviour, but it gives us a place to add other
      post-build actions.
      This also simplifies the code in Cmd{Configure,Build,Repl}.
      A few examples are mentioned in a comment, including updating:
        bin symlinks/wrappers
        haddock/hoogle/ctags indexes
        delete stale lib registrations
        delete stale package dirs
    • Duncan Coutts's avatar
      Go back to the plan.json reflecting only env inputs · 602a5434
      Duncan Coutts authored
      In particular it will not reflect the status of the build, ie it will
      list all the source packages and their configuration but not say if
      they're already installed in the store.
      There's a slightly vauge principle at work here, that we should
      distinguish between build outputs and build ststus: build outputs
      should be (morally) pure functions of the environment, where as build
      status is about the degree to which the current cached outputs reflect
      the state of the inputs (ie are build outputs up to date or not).
      So in this case that means the plan.json is considered an output and
      thus it should not depend on the state of the store. We can certainly
      provide machine readable status info, but this should be a separate
    • Herbert Valerio Riedel's avatar
      Comment out -fno-ignore-asserts in cabal.project · 0c99981f
      Herbert Valerio Riedel authored
      This was causing serious performance regression in the solver
      so I'm commenting thise one out; see #3911 for details.
      [skip ci]
  2. 28 Sep, 2016 2 commits
    • Herbert Valerio Riedel's avatar
      Add `Distribution.Simple.Utils.ShortText` type (#3898) · 993d20a2
      Herbert Valerio Riedel authored
      This implements a type with a compact representation of `[Char]`.
      The data is stored internally as UTF8 in an 'Data.ByteString.Short.ShortByteString'
      when compiled against `bytestring >= 0.10.4`, and otherwise in a
      plain old `[Char]`.
      `ShortByteString` is available only from `bytestring` 0.10.4 on, and GHC
      7.8.4 was the first GHC to bundle `binary-0.10.4`. So this fallback
      affects mostly only GHC 7.6 and older.
      Note: Originally a strict `ByteString` was used as fallback for this patch. However, the 
      `[Char]` fallback avoids pinned memory and may be more preferable when dealing with
      many small `ShortText`s
    • Herbert Valerio Riedel's avatar
      Make `Version` type opaque (#3905) · bb2026c4
      Herbert Valerio Riedel authored
      Similiar to dabd9d98 which made
      `PackageName` opaque, this makes `Distribution.Version.Version` opaque.
      The most common version numbers occuring on Hackage are 3- and
      4-part versions. This results in significant Heap overhead due to
      `Data.Version`'s inefficient internal representation.
      So like the `PackageName` commit, this commit is a preparatory commit to
      pave the way for replacing `Version`'s internal representation by a
      representation with a memory footprint which can be an order of
      magnitude smaller.
      Finally, this also adds a new functor-like convenience function
          alterVersion :: ([Int] -> [Int]) -> Version -> Version
      for modifying the version number components.
  3. 27 Sep, 2016 4 commits
    • Mikhail Glushenkov's avatar
      Merge pull request #3900 from arianvp/master · c70ae433
      Mikhail Glushenkov authored
      Make interactive setup delegate CTRL+C
    • Arian van Putten's avatar
      Make interactive setup delegate CTRL+C · 005f6dfa
      Arian van Putten authored
      Fixes #3899
      When cabal new-repl spawns ghci, it spawns this as a subprocess.
      In UNIX like systems, if a CTRL+C is sent to this child process
      it also bubbles up to the parent process, causing it to terminate.
      Also see
      However, this is not what we want for interactive subprocesses.
      Interactive processes usually define their own handlers for CTRL+C,
      for example GHCi uses CTRL+C to reset the input buffer. So instead
      of terminating on CTRL+C we want to delegate CTRL+C to GHCi
      and let it do its thing.
      Luckily, we can enable CTRL+C delegation such that the parent
      process ignores the CTRL+C and instead delegates it to the
      child process.
    • Herbert Valerio Riedel's avatar
      Make `PackageName` type opaque (#3896) · dabd9d98
      Herbert Valerio Riedel authored
      When looking at heap-profiles of `cabal-install`, the `(:)` constructor
      stands out as the most-allocated constructor on the heap.
      Having to handle 10k+ package names contributes to the allocation
      numbers, especially on 64bit archs where ASCII `String`s have a 24 byte
      per character footprint.
      This commit is a preparatory commit to pave the way for changing
      `PackageName`'s internal representation to something like
      `ShortByteString` (which is essentially a thin wrapper around primitive
      `ByteArray#`s which themselves have have an overhead of 2 words + one
      byte per ASCII character rounded up to nearest word) which would allow
      to reduce the memory footprint by a full order of magnitude, as well as
      reduce pointer chasing and GC overhead.
    • Edward Z. Yang's avatar
      Merge pull request #3901 from dcoutts/installplan-fixes · 2ccfce17
      Edward Z. Yang authored
      InstallPlan fixes and misc housekeeping
  4. 26 Sep, 2016 13 commits
    • Herbert Valerio Riedel's avatar
      Introduce `Distribution.Compat.Prelude.Internal` + WARNING · 4c730f58
      Herbert Valerio Riedel authored
      This unexposes `Distribution.Compat.Prelude` again, and adds
      an exposed `.Internal` module which reexports the `.Prelude` module with
      an attached module-level `WARNING` pragma.
      This way users are discouraged to use this in `Setup.hs` files as they'd
      see compile warning like:
          Foo.hs:19:1-55: warning: [-Wdeprecations]
            Module ‘Distribution.Compat.Prelude.Internal’:
              This modules' API is not stable. Use at your own risk, or better yet, use @base-compat@!
      In `cabal-install` we import the `.Internal` module exactly once, and
      use -Wno-deprecations to suppress that the warning of that single
    • Herbert Valerio Riedel's avatar
    • Herbert Valerio Riedel's avatar
    • Herbert Valerio Riedel's avatar
      Introduce `Distribution.Client.Compat.Prelude` · 82a22706
      Herbert Valerio Riedel authored
      This is supposed to become more or less a superset of Cabal's
      As a side-effect,t his exposes `Distribution.Compat.Prelude` from the
      Cabal library (which may be actually a good thing, as it may be useful
      module to Setup.hs writers).
    • Duncan Coutts's avatar
      Add InstallPlan.{toGraph,toMap,keys,keysSet} · 9b40b06c
      Duncan Coutts authored
      At least keysSet will be useful shortly. Others are just for good
      measure (though there's probably some cases where we could avoid
      converting to and from lists if we used toMap or toGraph).
    • Duncan Coutts's avatar
      Rename InstallPlan "index" stuff to "graph" · ec09376d
      Duncan Coutts authored
      This better matches the feeling of the thing now, and also better
      matches the rest of the naming.
    • Duncan Coutts's avatar
      Better errors when the InstallPlan invariant is violated · 12a39b68
      Duncan Coutts authored
      Same idea as a patch proposed by ezyang but with somewhat less
      infrastructure needed. We just use individual asserts for each part of
      the invariant, so the assert source location tells us which part was
      violated. Hopefully call stacks can also tell us in which function the
      invariant was checked.
    • Duncan Coutts's avatar
      Detailed comments on the InstallPlan processing invariant · 72d0a3c8
      Duncan Coutts authored
      What they mean informally, to augment the formal definitions.
    • Duncan Coutts's avatar
      Extend the InstallPlan processing invariant · 48db1c08
      Duncan Coutts authored
      Add two additional properties that we believe are true:
       - All immediate reverse deps of packges that are currently processing
         are not currently being processed (ie not in the processing set).
       - The failed set is upwards closed, i.e. equal to its rev dep closure
    • Duncan Coutts's avatar
      Fix an InstallPlan.failed assertion · 48a21cbf
      Duncan Coutts authored
      Remove the wrong assertion:
        all (`Set.notMember` failedSet)  (tail newlyFailedIds)
      This is wrong for much the same reasons as the part of the invariant
      (in the previous patch) was wrong.
      Suppose Q depends on P1 and P2 and the processing set intially is
      {P1, P2}. Now suppose that P1 fails, so the failure set becomes {P1,Q}
      and the new processing set is {P2}. Now suppose P2 also fails. Then the
      newlyFailedIds is [P2, Q] and so the tail is [Q]. Of couse Q is in the
      failure set already, in violation of the above assertion. But this is a
      perfectly reasonable scenario. It's just the assertion that is wrong.
    • Duncan Coutts's avatar
      Merge pull request #3887 from dcoutts/filemonitor-fixes · 649f41bb
      Duncan Coutts authored
      FileMonitor fix
    • Duncan Coutts's avatar
      Fix the InstallPlan Processing invariant · 21be7d9d
      Duncan Coutts authored
      Previously it required:
        noIntersection processingClosure failedSet
      Lets see what it's saying and why it's wrong. Suppose we have P1 and P2
      in the processing set. We have Q that depends on P1 and P2. So the
      processingClosure is {P1,P2,Q}. Now suppose building P1 fails. The
      failedSet is now {P1,Q1}, and the processingSet is {P2}. The
      processingClosure however is {P2,Q}. Thus we have a non-empty
      intersection between the processingClosure and failedSet, namely {Q}.
      So clearly it's bogus, but also it's not clear if I was thinking of
      something else that is correct. So we just drop this part of the
      This should fix issue #3889.
    • Duncan Coutts's avatar
      Merge pull request #3890 from dcoutts/installplan-installed-state · 3b6585ad
      Duncan Coutts authored
      Add InstallPlan.Installed state
  5. 25 Sep, 2016 2 commits
    • Duncan Coutts's avatar
      Add comment on MonitorStateFileSet being a bag [skip ci] · 2fb4de6c
      Duncan Coutts authored
      Explain why it's not represented by a Set.
    • Herbert Valerio Riedel's avatar
      Use hash-consing to optimise index cache (#3897) · e0dd63cc
      Herbert Valerio Riedel authored
      Without this optimisation, `cabal info somethingnonexisting` results in
           960,397,120 bytes allocated in the heap
           739,652,560 bytes copied during GC
            67,757,128 bytes maximum residency (24 sample(s))
             2,234,096 bytes maximum slop
                   147 MB total memory in use (0 MB lost due to fragmentation)
      with this optimisation:
         1,000,825,744 bytes allocated in the heap
           656,112,432 bytes copied during GC
            44,476,616 bytes maximum residency (24 sample(s))
             2,302,864 bytes maximum slop
                   109 MB total memory in use (0 MB lost due to fragmentation)
      So the total memory in use is significantly lower. The total runtime is
      also slightly reduced, from
        INIT    time    0.001s  (  0.001s elapsed)
        MUT     time    0.683s  (  1.050s elapsed)
        GC      time    0.946s  (  0.946s elapsed)
        EXIT    time    0.005s  (  0.005s elapsed)
        Total   time    1.637s  (  2.002s elapsed)
        INIT    time    0.001s  (  0.001s elapsed)
        MUT     time    0.664s  (  0.988s elapsed)
        GC      time    0.797s  (  0.797s elapsed)
        EXIT    time    0.004s  (  0.004s elapsed)
        Total   time    1.467s  (  1.789s elapsed)
      Note that there's currently ~80k cache entries, but only ~10k unique package names
      and ~6k unique versions. So hash-consing helps reduce the amount of heap objects 
      for both value types by one order of magnitude, which among other benefits also
      reduces GC overhead.
  6. 24 Sep, 2016 15 commits
    • Mikhail Glushenkov's avatar
      Merge pull request #3894 from hvr/pr/rtsopts · ecd2cb1f
      Mikhail Glushenkov authored
      Enable `-rtsopts` for `cabal` executable
    • Herbert Valerio Riedel's avatar
      Merge PR #3893 (Implement `--index-state` aka index freezing) · 1535bd3e
      Herbert Valerio Riedel authored
      This PR implements a new flag `--index-state` (and its `cabal.project`/config-file
      equivalent `index-state: ...`) which allows to change the source package index
      state the solver uses to compute install-plans. This is particularly useful in
      combination with freeze-files in order to also freeze the state the package
      index was in at the time the install-plan was frozen.
      This provides the core functionality, on which future enhancements can build
      upon. See also description of PR #3604 for some possible enhancements.
    • Herbert Valerio Riedel's avatar
    • Herbert Valerio Riedel's avatar
      Enable `-rtsopts` for `cabal` executable · 5edaa0c4
      Herbert Valerio Riedel authored
      Fwiw, the GHC executables are compiled with full `-rtsopts` as well.
      The benefit is that one can use flags such as `-M1G` to limit the heap
      space, or profile memory usage via `-hT`.
    • Herbert Valerio Riedel's avatar
    • Herbert Valerio Riedel's avatar
    • Herbert Valerio Riedel's avatar
    • Herbert Valerio Riedel's avatar
      Turn 'Timestamp' into a proper type (#3892) · 060b9061
      Herbert Valerio Riedel authored
      Most notably, this allows us to provide a custom instance for `Text`
    • Duncan Coutts's avatar
      Refactor implementation of InstallPlan.installed · d5288df0
      Duncan Coutts authored
      All the use sites (currently only two but soon to be three) use
      InstallPlan.installed to do a bulk change of states, differing only in
      the filter condition. So it simplifies things and shares more code if
      we make the main one be the bulk version. The InstallPlan.remove already
      works similarly.
    • Duncan Coutts's avatar
      Simplify plan improvement, avoid reading store ghc-pkg db · bf0b5dfe
      Duncan Coutts authored
      Install plan improvement is the process of replacing configured source
      packages with installed instances from the store. Originally we did this
      by reading the ghc-pkg db to get the InstalledPackageInfo for all the
      packages in the store. We had to do that because when we replaced
      configured source packages by installed instances we used the
      PreExisting constructor which requires an InstalledPackageInfo, which we
      get from the installed package db. But now that we no longer use
      PreExisting for packages from the store we also no longer need the
      InstalledPackageInfo. All we need is a set of UnitIds. Also, once
      support for depending on executables was added then we needed a way to
      do plan improvement for executable packages/components. We did this by
      the simple approach of grabbing the dir listing for the store and using
      that as a set of UnitIds.
      So this patch extends the approach we use for executables and uses it
      for all packages. This means we no longer load the package db for the
      Note that still need to create the package db in the store. Previously
      we would create it when getting the package db contents, but we don't
      do that any more, but we still need to make sure the db exists.
      This also relates to the locking protocol in the store. The goal for the
      store is to be able to access and update it concurrently. The locking
      protocol will include checking membership by checking if the directory
      entry for the package is present. So this patch gets us to the right
      point for the reading side, leaving the writing side to do.
    • Duncan Coutts's avatar
      Distinguish PreExisting vs Installed in BuildStatus · 81691044
      Duncan Coutts authored
      Not a big deal, but should be useful later for more precise status
      reporting. For now just means the rebuild reasons can be more precise.
    • Duncan Coutts's avatar
      Remove now-unused InstallPlan.preexisting · f6feb199
      Duncan Coutts authored
      We only ever switch Configured to Installed. The PreExisting state only
      comes from the original solver plan, which only uses installed packages
      from the global db.
    • Duncan Coutts's avatar
      Start using new InstallPlan.Installed state · 0de4177c
      Duncan Coutts authored
      Change improvement and --dry-run phases to use Installed state rather
      than the PreExisting state. This means that PreExisting is now only used
      for installed packages from the global db, and never for installed
      packages from the store.
    • Duncan Coutts's avatar
      Update encodePlanAsJson for Installed package state · 9d6205e6
      Duncan Coutts authored
      The change in how we use the PreExisting vs Installed states means that
      we'll now have full details for all packages, rather than installed
      ones having only the subset of info available from the
      InstalledPackageInfo. So the 'type' field now can take the values
      "pre-existing", "configured" or "installed".
      Also do a little bit of tidying up.
    • Duncan Coutts's avatar
      Add an Installed state to InstallPlan packages · 435725ef
      Duncan Coutts authored
      This patch just adds the state without yet using it. That'll follow in
      subsequent patches.
      So why add an Installed state? Didn't we just remove the Installed,
      Processing and Failed states? Those states were used when we followed
      the approach of updating the InstallPlan as a build progressed (whereas
      we now do traversals without altering the InstallPlan).
      The idea of adding an Installed state now is that we can more usefully
      represent the state of the plan when we "improve" the plan with packages
      from the store or when we update the plan having checked if inplace
      packages are up to date. Currently in these two places we replace
      Configured source packages with PreExisting packages. There's a couple
      problems with this. Firstly the PreExisting state only contains an
      InstalledPackageInfo which means we loose information compared to all
      the detail in the Configured source package. This is relevant for things
      like plan.json output or other features that want to know the status of
      a project. Secondly we have to fake things for executables since they
      are not properly represented by InstalledPackageInfo.
  7. 23 Sep, 2016 1 commit
    • Duncan Coutts's avatar
      New Rebuild monad utils and use in ProjectPlanning · c7d55c3c
      Duncan Coutts authored
      Monitored variants of createDirectory and getDirectoryContents.
      Define thee wrappers in the RebuildMonad module so we have fewer
      open-coded tricky monitorFiles calls.
      In particular replace a glob monitor on the content of the store with a
      monitor on the store directory itself. This is valid based on the
      behaviour of directory mtimes, which is specified by posix and we have a
      sanity check for it in the unit tests.