GitLab

Convert hadrian buildscripts to unsuffixed, dashed form

final cleanups

Fixes #17612 by adding a `cabal.configure.opts` key for
`hadrian.settings`.

Throw an error if the user requests a flavour for which there is more
than one match.

Fixes #17156.

Currently, if you change these ^ flavour parameters, rebuilding is not
triggered, since `programContext` doesn't set up a dependency on
those values.

Exposing these values via an oracle does set the dependency and
properly triggers a rebuild of binaries.

Several attempts to factor out these actions ended up in cyclic
dependency here or there. I'm not absolutely happy with this variant
either, but at least it works.

====

Issue repro:

In UserSettings.hs:
```
dbgDynamic = defaultFlavour { name = "dbg-dynamic"
                            , dynamicGhcPrograms = pure True,
                            ... }

dbgStatic = defaultFlavour { name = "dbg-static"
                           , dynamicGhcPrograms = pure False
                           ... }
```

Then in console:

```
$ hadrian/build.sh -j --flavour=dbg-dynamic
... does the build
$ hadrian/build.sh -j --flavour=dbg-static
... does nothing, considers binaries up to date
```

They take the general form `foo.bar.baz [+]= some values`, where
`=` completely overrides the arguments for a builder and `+=` extends
them. We currenly only support settings for updating the GHC and C
compiler options, of the form:

```
  {stage0, ..., stage3 or *}.{package name or *}
                            .ghc.{c, hs, link, deps, toolargs or *}.opts

  {stage0, ..., stage3 or *}.{package name or *}
                            .cc.{c, deps or *}.opts
```

The supported settings and their use is covered in the new section
of `hadrian/doc/user-settings.md`, while the implementation is explained
in a new Note [Hadrian settings].

Most of the logic is implemented in a new module, `Settings.Parser`, which
contains key-value assignment/extension parsers as well as utilities for
specifying allowed settings at a high-level, generating a `Predicate` from
such a description or generating the list of possible completions for a given
string.

The additions to the `Settings` module make use of this to describe the
settings that Hadrian currently supports, and apply all such
key-value settings (from the command line and `<root>/hadrian.settings`)
to the flavour that Hadrian is going to proceed with.

This new setting system comes with support for generating Bash completions,
implemented in `hadrian/completion.sh` and Hadrian's `autocomplete` target:

> source hadrian/completion.sh
> hadrian/build.sh stage1.base.ghc.<TAB>
stage1.base.ghc.c.opts     stage1.base.ghc.hs.opts
stage1.base.ghc.*.opts     stage1.base.ghc.deps.opts
stage1.base.ghc.link.opts  stage1.base.ghc.toolargs.opts

Running the `./hadrian/ghci` target will load the main compiler into
a ghci session. This is intended for fast development feedback, modules are only
typechecked so it isn't possible to run any functions in the repl.

You can also use this target with `ghcid`.

The first time this command is run hadrian will need to compile a few dependencies
which will take 1-2 minutes. Loading GHC into GHCi itself takes about 30 seconds.

Internally this works by calling a new hadrian target called `tool-args`.
This target prints out the package and include flags which are necessary
to load files into ghci. The same target is intended to be used by other
tooling which uses the GHC API in order to set up the correct GHC API
session. For example, using this target it is also possible to use HIE
when developing on GHC.

Test Plan:
Ensure build environment does NOT have a system libffi installed (you may want to use a nix environment).
Then `hadrian/build.sh -c --flavour=default`

Reviewers: bgamari

Subscribers: rwbarton, carter

GHC Trac Issues: #15837

This is a preparation for #16295: https://ghc.haskell.org/trac/ghc/ticket/16295

This commit mostly focuses on getting rid of untracked dependencies,
which prevent Shake's new `--shared` feature from appropriately caching
build rules.

There are three different solutions to untracked dependencies:

* Track them! This is the obvious and the best approach, but in some
  situations we cannot use it, for example, because a build rule creates
  files whose names are not known statically and hence cannot be
  specified as the rule's outputs.

* Use Shake's `produces` to record outputs dynamically, within the rule.

* Use Shake's `historyDisable` to disable caching for a particular build
  rule. We currently use this approach only for `ghc-pkg` which mutates
  the package database and the file `package.cache`.

These two tickets are fixed as the result:

Ticket #16271: ​https://ghc.haskell.org/trac/ghc/ticket/16271

Ticket #16272: ​https://ghc.haskell.org/trac/ghc/ticket/16272 (this one
is fixed only partially: we correctly record the dependency, but we
still copy files into the RTS build tree).

This adds a handful of LLVM flavours and the accompanying
documentation. These flavours are mostly uninteresting, but
exist in the Make system.

This patch introduces the `userDefaultFlavour` user setting. It should
be the name of the default flavour to use when no --flavour argument is
passed. Before this patch, we would just always default to... the
`default` flavour. With this patch, we default to whatever Flavour whose
name is `userDefaultFlavour`, therefore providing a way for users to
"persist" their choice of flavour, not having to repeat --flavour=[...]
in every hadrian command.

Test Plan:
Set `userDefaultFlavour = "quickest"`, run `hadrian/build.sh`, check
that the quickest flavour is indeed picked.

Reviewers: snowleopard, bgamari

Reviewed By: snowleopard

Subscribers: mpickering, rwbarton, carter

GHC Trac Issues: #15890

Differential Revision: https://phabricator.haskell.org/D5454

Sadly subtrees haven't worked quite as well as we would have liked for
developers. See Hadrian #440.