Backport #10731: cabal-install: Be less eager to configure external programs (#10875)

* Revert "cabal-install configureCompiler: configure progdb"

This reverts commit 8bdda9c0.

In configureCompiler the call to configureAllKnownPrograms was too
eager. When called it selected the version of tools from PATH (such as
alex), and then when a package was configured these tools were passed
using `--with-alex` options to configure, which meant that the
build-tool-depends versions were not used. (See #10692)

Why was this call introduced in the first place? Because
configureCompiler would a different result depending on whether:

* It is run for the first time, the `ProgramDb` will contain
  unconfigured programs.
* It is run subsequently, `ProgramDb` is read from disk, it does not
  contain unconfigured programs.

Reverting this commit rexposes the bug that the serialised ProgramDb
will not contain UnconfiguredPrograms (in the case where reconfiguring
is avoided).

However, there are no code paths which require this logic in
`cabal-install` currently. The configuration phase happens again each
time that `Cabal` is called, with a populated `ProgramDb`. We will
fix this before the next major `cabal-install` release, but it would not
be suitable to backport.

In the future we will fix this properly by refactoring
`configureCompiler` so that `ProgramDb` is not serialised. The general
approach will be to make `configCompilerEx` return a pair of configured
programs (`ghc` and `ghc-pkg`) and expose an additional function from
`Cabal` which uses these two programs to perform the modifications to
the `ProgramDb` which `configCompilerEx` performs.

Also see #2238 and #9840

Fixes #10692

(cherry picked from commit 1c64bb8c)

# Conflicts:
#	cabal-testsuite/PackageTests/ExtraProgPath/setup.out

* Add a test to check that build-tool-depends are used (#10692)

The testcase is not so easy to write because

* The bug only surfaces when the build-tool you are depending on is
  known (ie alex, happy etc)
* But then it is tricky to write a test, as we can't depend on the known
  tools or bundle the source for them.
* So we create a fake "alex", which cabal then invokes on a fake ".x"
  file. This is maybe a bit fragile if the way cabal invokes alex
  changes in future, but then the test can be modified as well.

Ticket #10692

(cherry picked from commit 24f83951)

* Add a test to check that extra-prog-path is honoured for local packages

Whilst fixing #10692, I realised there was also this bug where
extra-prog-path would not be honoured for specific packages.

The idea behind extra-prog-path is that each local package can use a
different version of a preprocessor if desired.

(cherry picked from commit 2c19bf35)

* fixup! fix conflict

---------

Co-authored-by: Matthew Pickering <matthewtpickering@gmail.com>
Co-authored-by: Artem Pelenitsyn <a.pelenitsyn@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>