WIP: Implement Multiple Home Units for GHC (#10827)

Thank you for your contribution to GHC!

Please take a few moments to verify that your commits fulfill the following:

• are either individually buildable or squashed
• have commit messages which describe what they do (referring to [Notes][notes] and tickets using #NNNN syntax when appropriate)
• have added source comments describing your change. For larger changes you likely should add a [Note][notes] and cross-reference it from the relevant places.
• add a testcase to the testsuite.
• replace this message with a description motivating your change

Motivation

The main motivation can be found in #10827 (closed).

Structure

The MR is split into multiple commits that are all individually buildable (although not all pass the testsuite).

There are two commits that are should not be merged with this MR:

• Modify hie.yaml to only load the compiler package
  • Modifications for hie.yaml as I saw fit for developing GHC code-base. Can be removed, once approved.
• Debug Output
  • Adds debug output that helped me debugging. It is not intended that this commit is merged.

This MR also subsumes !935 (closed) and includes the changes from it in the commit:

• Implement basics for multi-unit support in HscEnv
  • If desired, this commit can be integrated in the main changes entirely. Changes are squashed, as they are individually not useful.

The main changes of this MR are in the commits:

• Design API for multiple home units
  • Design an API for end-users and GHC to work with multiple home-units.
• Implement multiple home unit support
  • Main changes. Make GHC work with Multiple Home Units
• Implement GHCi interface for multiple home units
  • Purely interface changes for GHC and GHCi.
  • This MR re-implements clumsily !3897 (merged), I would prefer if that MR gets merged and this MR is rebased on top it.

UI Design and general comments have been added to the wiki multiple-packages-per-session page.

Open Issues

A few issues that need to be taken care of:

• There are quite a lot of "TODO:" lines
• Remove or fix the commit "Dubious commit, review each line"
  • It is the result of three months of rebasing and changing code.
• Test-coverage is far from sufficient. At least the following test-cases must be covered:
  • Produce compiler artifacts with multiple home units
  • Produce compiler artifacts with multiple home units using only a singleton home units (should have same behaviour as --make)
• Discuss design decisions regarding GHC UI.
• Discuss how to handle statefulness in GHC API.
  • In particular, prefer explicit UnitId parameters over implicit changes to HscEnv and vice versa.
• Maybe individual commits need more explanations.

Testing in the wild

Testing this MR in the wild is doable with a fork of cabal and HEAD.Hackage. Example:

$ cabal init --libandexe --no-comments --source-dir=src --application-dir=app -p example
$ $HEADHackage/scripts/head.hackage.sh init
$ cabal update
$ cabal repl lib:example exe:example --allow-newer=base,ghc-prim

---

If there is something missing for this MR, I happy to add it.

cc @Ericson2314, @mpickering, @hsyl20