Cleanup around subprocess helpers

Fix concurrency/exception bugs in asyncFetchPackages

If `withTempFileName` receives an asynchronous exception (e.g.
a canceled async), the bracket cleanup handler will attempt to
remove the temporary file. This can fail with an IO exception.

Regular bracket then throws that IO exception, swallowing the
asynchronous exception. To calling code, this appears no different
from an IO exception thrown from the body, and it won't be able
to tell that it should be exiting promptly.

This manifests concretely during temporary file clean-up of
`asyncFetchPackages` on Windows (seen during unit testing in CI),
where temporary file removal fails (on GHC 8.4), which leads
to `concurrently` failing to cancel the outstanding download
because it's handled like a regular download failure.

This fixes deadlock in asyncFetchPackages:

  withAsync action1 (\_ -> action2)

loses exceptions thrown by action1. If in addition, action2
has a dependency on data produced by action1 (as is the case
here), this will block indefinitely.

(This bug caused some of the new tests to hang, since after
the change to use async-safe try, it became easier for
fetchPackages to throw an exception.)

This is a step towards fixing interrupt handling during
asynchronous download.

The concrete bug fixed by this changes is  that `try` catches
asynchronous exceptions, preventing `withAsync` from interrupting
it when its `body` action finishes prematurely.

This manifests during `cabal build` when hitting Ctrl-C during
download, e.g.:

1. Ctrl-C interrupts a curl process
2. This interrupt is converted into a UserInterrupt exception
   via the process package's delegate_ctlc functionality.
3. UserInterrupt bubbles up through `fetchPackage`, is caught by
   the `try` (fine) and writen to the result mvar.
   Meanwhile, `fetchPackage` continues its loop, starting to
   fetch the next package.
4. Some `rebuildTarget` is waiting for the interrupted download;
  `waitAsyncFetchPackage` rethrows UserInterrupt in that thread.
5. UserInterrupt bubbles up through `collectJob`, `execute`, the
   `body` action.
6. `withAsync`'s finalizer cancels the `fetchPackages` async.
7. The `fetchPackages` async receives the AsyncCancelled exception
   while fetching the next package (see 3. above). This interrupts
   the download action (it shouldn't matter whether we happen to
   be running curl again or not), and AsyncCancelled bubbles up
   through `fetchPackage`, is caught by the `try` (not fine!) and
   written to the mvar. But no-one is reading that mvar anymore
   because the build job already aborted (step 5), and that
   AsyncCancelled exception is lost.
8. `fetchPackages` keeps doing its thing, exiting eventually.

Note that this change affects both instances of `try` in this story:
`UserInterrupt` is also an asynchronous exception, so after the
change the `UserInterrupt` takes a different path from step 3.
Followup commits fix bugs along those paths.

This primarily trigger concurrency bugs that are fixed in the
follow-up commits.

- Sort modules and tests alphabetically (they were maddeningly close to alphabetic)
- Consistently label unit test groups at top-level

* don't crash on a few stray exceptions

* try -> catch and display

* act on reviewer comments

Co-authored-by: Gershom Bazerman <gershom@arista.com>

- Set enable_process_jobs on a variant of System.Process.proc
  instead of just for System.Process.createProcess
- In Distribution.Simple.Utils, only use this proc instance.
- Replace use of printRawCommand* by a unified helper logCommand,
  and use this more consistently. The output format is changed
  slightly.
- New helpers rawSystemProc{,Action} for use with new proc

Aside from the logging changes, this should be a no-op.

Try to resolve unused-packages warnings

Marius Melzer authored 3 years ago and Mikolaj Konarski committed 3 years ago

The documentation for the active-repositories option "none" was incorrect as it missed the colon.

* Avoid templating GitHub Actions workflow

GitHub Actions workflow have sufficient power to express what we need.
We don't need to maintain and additional templating solution on top.

* Add GHC 9.2, bump bounds, fix syntax

* Switch to official haskell image

* Always run cli tests for ghc 8.2 and above

* Remove step to regenerate GitHub Actions workflows

* Fix missed reference to GHC version

* Fix yaml syntax

* Fix type in the GHC version

* More CI changes

- Run bootstrap.yml on ubuntu-latest
- Use explicit matrix for linux.yml
- Drop containers in favour of haskell setup action
- Drop workaround for ancient git

* Remove unneeded package from CI setup

I belive this is only necessary to run `cabal man` which we do not in
the CI.

* Drop old GHCs from the CI

* Switch macos.yml to haskell/action/setup

Also add the same GHC versions as Linux.

* Simplify CI

- Remove cabal-plan, we actually never call it (I think)
- Remove vendored cabal-doctest
- Remove few stray allow-newer clauses no longer necessary, apparently

Originally done by @gbaz in PR #7907.

* Mark GHC 9.2.1 as experimental

* Remove reference to cabal-plan from validate.sh

* setup-haskell action already runs cabal update

* Add missing build matrix in test-windows-dogfood

* Replace cabal-plan list-bin with cabal list-bin

* Enable caching in the CI

* Fix typo

* Remove continue-on-error until I figure it out

* Keep naming consistent

* Temporarily disable 8.0.2 on macos

* Add missing step id

* Tweaks

Remove workaround for nektos/act, it accidentally sneaked in.

* More tweaks

* Tweaks

* Restore cabal-plan, temporarily mark everything experimental

cabal list-bin doesn't seem to work like cabal-plan does.

* Tweaks

* Ensure cabal-plan executable gets built

* Install automake on MacOS

* Tweaks

* Tweaks

Link experimental flags to relative GitHub issues

* Fix typo

Remove deprecated file finding functions

Reduce default verbosity for running scripts

Closes #5508

Add a doc section on cabal list-bin.