Matthew Pickering authored 2 years ago and Francesco Gazzetta committed 1 year ago

There are several parts to this patch which are logically distinct but
work together to support the overal goal of starting a GHCi session with
multiple packages loaded at once.

1. When a user writes "cabal repl <target>" then if the user is using a
   compiler > ghc-9.4.* then we will attempt to start a multi-session
   which loads the selected targets into one multi-package session of
   GHC.
1a. The closure property states that in order to load components `p` and `q` into
    the same session that if `p` depends on `z` and `z` depends on `q`
    then `z` must also be loaded into the session.
1b. Only inplace packages are able to be loaded into a multi session (if a component
    `z` exists then it is already made into an inplace package by
    cabal). Therefore cabal has already engineered that there is source
    code locally available for all packages which we will want to load
    into a session.

2. The solver is unmodified, the solver is given the repl targets and
   creates a build plan as before. After the solver is completed then in
   `setRootTargets` and `pruneInstallPlan` we modify the install plan to
   enforce the closure property and mark which dependencies need to be
   promised.

   * Mark the current components as `BuildInPlaceOnly InMemory`, which
     indicates to the compiler that it is to be built in a GHC
     multi-session.
   * Augment the component repl targets to indicate that components
     required by the closure property (in addition to normal targets)
     will be loaded into the repl.
   * Modify the dependency edges in `compLibDependencies` to indicate
     which dependencies are the promised ones (which is precisely
     components which are `BuildInPlaceOnly InMemory` build styles).
     This is the field which is eventually used to populate the
     `--dependency` argument to `./Setup configure`.

Fixes #8491

* Use shorter hash for script-builds directories

Using a Base64 hash and truncating it to 26 characters, saves 38 chars,
which helps avoid long paths issues on Windows, while still providing
130 bits of hash in order to avoid collisions.

Bug #8841

* Use the script cache dir as the dist dir

Putting script build products under dist-newstyle within the cache
directory is unnecessary because we already control the cache directory
and can ensure there are no conflicts.

* Use the actual script name in the executable name

Previously, the script name was sanitized in final executable name,
because the executable name had to match the component name, which only
allowed for a limited character set. Now we can use the actual script
name in the executable name. This only lets us shorten the component
name without losing clarity.

* Add changelog entry

* Reenable script tests for Windows/ghc-9.4.*

Partially revert 78cbeba1.

* do not print "up to date" unless running build

* Update cabal-tests

* Add changelog

* Remove accidentally added test generated files

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

We've seen some instability in the test, compare

  https://github.com/haskell/cabal/issues/8416

This adds timestamped output to make it easier to
see what goes wrong in case the test fails again.

This makes the `finally` apply to the last `do` block as intended.
No real effect on the test, though.

* Turn off T3827 for new GHCs due to heisenbugs not caused by cabal

* Disable the test totally on Linux until we stop taking GHC from GHA

See https://github.com/haskell/cabal/issues/8032#issuecomment-1204113187

* It failed on OSX now, so let's disable it everywhere except on Windows

Who would have thought.

+ Update file-format-changelog.rst

The "up and running" file was written before the signal handler
was installed, so it was possible for the test to kill the
subprocess before, leading to a missed "exiting" line in the
output.

I hope this fixes the sporadic failures we're seeing (#8080).

Failures are tracked in issues
  https://github.com/haskell/cabal/issues/7610
  https://github.com/haskell/cabal/issues/7987
  https://github.com/haskell/cabal/issues/7989
  https://github.com/haskell/cabal/issues/8028
  https://github.com/haskell/cabal/issues/8032

* Pass such scripts with: -x hs script
* Disable extension check for scripts
* Disable running preprocessors for scripts because they're based on
  extension
* Add test case for script with no extension

Resolves #7910

* Add support for script build caching to cabal run

Enable caching of script builds by changing the location of the fake
package directory from a tmp directory to:
<cabal_dir>/scipt-builds/abs/path/to/script/

Resolves: #6354
WIP: #7842

* Add support for scripts to cabal build.

Added module Distribution.Client.ScriptUtils for code to deal with
scripts that is common between commands.

WIP: #7842

* Add script support to cabal clean.

This changes the behaviour of cabal clean to accept extra args, which it
now interprets as script files. The behaviour of cabal clean is the same
when given extra args. When given extra args it instead removes the
caches for those scripts and also any orphaned caches (caches for which
the script no longer exists)

In addition this commit changes the cache to use hashes of paths because
this significantly simplifies the implementation of clean, and more
importantly it prevents collisions when a script has the name of the
subdirectory of a previously cached script.

WIP: #7842

* Add script support to cabal repl

repl starts in the correct directory and points directly to rather than
a dummy, so that reloading works properly.

There is a downside to the current approach which is that it uses a
different fake-project.cabal file from run and build, so it cannot share
the same cache with them.

WIP: #7842
WIP: #6149

* Added changelog for pr #7851

* Fix `cabal run script.hs` issue with --builddir

Fixes tests:
cabal-testsuite/PackageTests/NewBuild/CmdRun/Script/cabal.test.hs
cabal-testsuite/PackageTests/NewBuild/CmdRun/ScriptLiterate/cabal.test.hs

* Fixes for `build script` and `repl script`

- Fix build issue introduced in 079c5f0e, where build was being passed
the wrong target filter
- Fix repl issue where script didn't work in a project context.
- Refactor code to share logic between repl and build/run
- Ensure temp directories are only created when needed

* Bug fixes relating to script support

ScriptUtils:
- Hash prefix for cache dirs was applied incorrectly.
- Overwriting fake-package files causes repeated work in some cases.
CmdClean:
- Clean distdir for script when --builddir is passed
- Always clean orphans because because there is no good way to specify
they should be cleaned. This may be bad behaviour in some obscure cases
(a cache is temporarily orphaned and an unrelated clean is run), but at
worst results in a cache rebuild.

* Add tests for improved script support

- Basic script support for build/repl/clean which checks for cached
project files
- Add check for cached project files to basic run script test
- No repeated work for build/build, build/run, run/run, and repl/repl
- Clean does not remove cache for existing scripts
- Clean does remove orphaned script caches

* Fix clean bug uncovered by 5fad1214

- clean was trying to read source-builds even if it didn't exist
- add test specific to this case with other clean tests

* Update documentation for better script support

Ready for review: #7851
May close: #7842, #6354, #6149

* Attempt to fix `repl script` on Windows

PR #7851

* Attempt to fix remote test failures

Test logs showed that the failures where because the tests depended on a
module from cabal-install that some ghc versions could not find.

Instead of depending on cabal-install, I copied the needed function into
Test.Cabal.Prelude (It seemed like an acceptable place for it)

PR #7851

* Attempt to fix `repl script` on Windows

PR #7851

* Attempt to fix tests on old ghc versions

Tests failing on pre-AMP ghcs due to unsanctioned use of (<$>)

PR #7851

* Feedback: Update docs and formatting

PR #7851

* Feedback: code style changes

- remove partial selectors
- make a constant for fake-package.cabal

PR #7851

* Feedback: make hidden control flow explicit

PR #7851

* Feedback: add expected fail script run tests

PR #7851

* Fix `repl script` when cwd is deeper than cachedir

PR #7851

* Use script in-place for build or run

- Set the hs-source-dir to the location of the script for build and run,
  the same as with repl
- This removes the need to copy the script
- repl no longer needs a separate cache because all three commands
  use identical project files
- Adds multi-module support to scripts for free (#6787)
- Add new build/repl test and run multi-module test

PR #7851

* Fix file-locking issue on Windows

PR #7851

* Fix script recompilation based on cwd

- Pass info about cwd to repl through --repl-options instead of hacking
  it into the project file.
- Improve paths output by makeRelativeCanonical, makeRelativeToDir, and
  makeRelativeToCwd.
- Script multi-module support works, but with warning in repl.
- Remove script multi-module mention support in docs.

PR #7851

* Make `repl script` respect --repl-no-load

* Feedback: minor refactor

Move argument truncation from targetStrings out of
withScriptContextAndSelectors to runAction

PR #7851

* Feedback: refactor and comments for repl options

PR #7851

* Don't use hs-source-dirs for scripts.

- instead pass absolute path to script in main-is
- resolves issue with relative paths on Windows
- simplifies code and gives prettier build output
- update tests because build output has changed
- removes ability to use multi-module scripts
  (which was never officially endorsed)
- remove test for multi-module scripts
- add checks for unsupported fields in scripts

PR #7851

* Update changelog for PR #7851

- Place "Error:" inside wrappers

  At least for the sake of the testsuite, the "Error:" prefix must be
  inside the bracket BEGIN...END that is added to the Cabal output
  during testing.

- Heuristic to not put "Error:" when already present

  This is because backpack outputs errors in an aligned way (Doc) where
  "Error:" is the introducing line.  Removing this breaks the layout.

- Remove manual "Error:" prefix where `die` adds it automatically.

- Update golden values.

I introduced TestCode which allows to report more status information
than bare ExitCode.