GitLab

This is due to the failures documented in #16087. The condition here could be
improved as it matches on `BUILD_FLAVOUR` instead of looking at the compiler
flags. However, it's better than nothing and I hope we will be able to fix these
issues before long.

This patch introduces the `userDefaultFlavour` user setting. It should
be the name of the default flavour to use when no --flavour argument is
passed. Before this patch, we would just always default to... the
`default` flavour. With this patch, we default to whatever Flavour whose
name is `userDefaultFlavour`, therefore providing a way for users to
"persist" their choice of flavour, not having to repeat --flavour=[...]
in every hadrian command.

Test Plan:
Set `userDefaultFlavour = "quickest"`, run `hadrian/build.sh`, check
that the quickest flavour is indeed picked.

Reviewers: snowleopard, bgamari

Reviewed By: snowleopard

Subscribers: mpickering, rwbarton, carter

GHC Trac Issues: #15890

Differential Revision: https://phabricator.haskell.org/D5454

Adds a `-fenable-ide-info` flag which instructs GHC to generate `.hie`
files (see the wiki page:
https://ghc.haskell.org/trac/ghc/wiki/HIEFiles).

This is a rebased version of Zubin Duggal's (@wz1000) GHC changes for
his GSOC project, as posted here:
https://gist.github.com/wz1000/5ed4ddd0d3e96d6bc75e095cef95363d.

Test Plan: ./validate

Reviewers: bgamari, gershomb, nomeata, alanz, sjakobi

Reviewed By: alanz, sjakobi

Subscribers: alanz, hvr, sjakobi, rwbarton, wz1000, carter

Differential Revision: https://phabricator.haskell.org/D5239

This was supposed to be working already but didn't work when we
specified several tests with --only. This patch not only fixes this
but also makes it possible to specify a subset of tests to run with the
TEST environment variable, like the make build system. Here are some
examples:

    hadrian/build.sh test --only=plugins01
    hadrian/build.sh test --only="plugins01 plugins02"
    TEST="plugins01 plugins02" hadrian/build.sh test
    TEST=plugins03 hadrian/build.sh test --only="plugins01 plugins02"

When both the TEST environment variable and the --only flag are used,
we simply concatenate the list of tests from both sources and ask the
testsuite driver to run them all.

This patch addresses #16026.

Test Plan: hadrian/build.sh test --only="plugins01 plugins02"

Reviewers: bgamari, snowleopard

Reviewed By: bgamari, snowleopard

Subscribers: rwbarton, carter

GHC Trac Issues: #16026

Differential Revision: https://phabricator.haskell.org/D5431

This patch introduces (phony) build targets of the form

    (1) stage<N>:<lib>:<name>   (e.g: stage1:lib:Cabal)
    (2) stage<N>:<exe>:<name>   (e.g: stage2:exe:ghc-bin)

where (1) builds the given library with the stage N compiler and (2)
builds the given executable with the stage N-1 compiler. This patch may
be generating too many such targets but it's a first stab that we can
refine.

This fixes #15949.

Test Plan: hadrian/build.sh stage1:exe:ghc-bin

Reviewers: bgamari, snowleopard

Reviewed By: bgamari

Subscribers: rwbarton, carter

GHC Trac Issues: #15949

Differential Revision: https://phabricator.haskell.org/D5434

Building a dynamically linked ghc is broken do to incorrectly building
and installing libffi. This disables building a dynamically linked ghc
and ghc-iserv-dyn while keeping most of the code in the relevant
commits: 79d5427e and 89fa34ec

Test Plan:
Ensure build environment does NOT have a system libffi installed (you
may want to use a nix environment).
Then `hadrian/build.sh -c --flavour=default`.

Reviewers: bgamari, alpmestan

Reviewed By: alpmestan

Subscribers: rwbarton, carter

GHC Trac Issues: #15837

Differential Revision: https://phabricator.haskell.org/D5430

Quickest does not build anything in a dynamic way, so that
flavour should default to not building "ghc programs" dynamically.

I ran into this problem by building the 'docs' target with quickest,
which failed because at some point we run a haddock command in a
context whose Way is sensitive to dynamicGhcPrograms, and it
started looking for dyn interface files when it shouldn't have.

Test Plan: hadrian/build.sh --flavour=quickest docs

Reviewers: snowleopard, bgamari

Reviewed By: snowleopard

Subscribers: rwbarton, carter

Differential Revision: https://phabricator.haskell.org/D5419

Following what was done to Rules.Library some time ago and to
Rules.Compile recently (D5412), this patch moves more rules away from
the "enumerate a lot of contexts and generate one rule for each" style
and instead uses the "parse data from file path to recover context"
approach. In fact, the only rules left to convert seem to be the ones
from Rules.Generate.

This effectively decreases the pauses described in #15938 further as
well as the amount of allocations and GC that we do, unsurprisingly.
Nowhere as drastically as D5412, though.

Test Plan: perform full build and generate docs

Reviewers: snowleopard, bgamari

Reviewed By: snowleopard

Subscribers: rwbarton, carter

GHC Trac Issues: #15938

Differential Revision: https://phabricator.haskell.org/D5422

Previously, as reported in #15938, resuming a build "in the middle",
e.g when building _build/stage1/libraries/base/, hadrian would take up
to a whole minute to get started doing actual work, building code.

This was mostly due to a big enumeration that we do in Rules.hs, to
generate all the possible patterns for object files for 1) all ways, 2)
all packages and 3) all stages. Since rule enumeration is always
performed, whatever the target, we were always paying this cost, which
seemed to grow bigger the farther in the build we stopped and were
resuming from.

Instead, this patch borrows the approach that we took for Rules.Library
in https://github.com/snowleopard/hadrian/pull/571, which exposes all the
relevant object files under as few catch-all rules as possible (8 here),
and parses all the information we need out of the object's path.

The concrete effect of this patch that I have observed is to reduce the
45-60 seconds pause to <5 seconds. Along with the Shake performance
improvements that Neil mentions in #15938, most of the pause should
effectively disappear.

Reviewers: snowleopard, bgamari, goldfire

Reviewed By: snowleopard

Subscribers: rwbarton, carter

GHC Trac Issues: #15938

Differential Revision: https://phabricator.haskell.org/D5412

Summary:
This is the latest in the 'findPtr' saga. See

  * 900c47f8
  * 561748cb

for the previous attempts. The problem with re-using the 'debug'
cabal flag for the purpose of forcing inclusion of 'findPtr' occurs
when 'debug' is one of the RTS ways, but RTS is not being compiled
with '-DDEBUG':

  * the 'debug' flag gets passed to cabal, signalling to build
    'rts' with the debug flavour, but also forcing inclusion of
    the 'findPtr'/'_findPtr' symbol

  * since '-DDEBUG' isn't enabled, that symbol doesn't show up in
    the libraries, so executable that depend on 'rts' (everything)
    will end up always requiring 'findPtr'/'_findPtr' but 'rts' won'y
    provide it!

The fix is simple: create a a new 'find-ptr' cabal-flag whose only
purpose is forcing '-Wl,-u,findPtr'/'-Wl,-u,_findPtr'. Then, enable that
flag when the RTS is being compiled with '-DDEBUG'

Test Plan: ./hadrian/build.sh -c # on mac

Reviewers: alpmestan, snowleopard, bgamari, erikd, simonmar, Phyx

Reviewed By: alpmestan, snowleopard, Phyx

Subscribers: Phyx, rwbarton, carter

GHC Trac Issues: #15956

Differential Revision: https://phabricator.haskell.org/D5404

See https://ghc.haskell.org/trac/ghc/ticket/15951

Iserv is not supported on Windows. This fixes #15950 but this whole situation
feels awfully fragile to me.

* (#15837 point 5) Use the -rpath gcc option and using the $ORIGIN
variable which the dynamic linker sets to the location of the ghc
binary.
* (#15837 point 4) "-fPIC -dynamic" options are used when building ghc
when either ghc or the rts have a dynamic way.
* (#15837 point 7) "-shared -dynload deploy" options are only used when
linking a library (no longer when linking a program).

Reviewers: bgamari, alpmestan

Reviewed By: alpmestan

Subscribers: adamse, rwbarton, carter

Differential Revision: https://phabricator.haskell.org/D5281

Previously, Hadrian was building all the appropriate dynamic ways for
libHSrts
but they were not picked up and installed in the package database when
we register the rts library. Since we use Cabal for registering
packages and
the .cabal files of packages as sources of truth for configuring and
installing,
we ended up patching Cabal to add a new field,
'extra-dynamic-library-flavours',
to specify those extra flavours to install in .cabal files:

    https://github.com/haskell/cabal/pull/5606

We now make use of this in rts.cabal.in to expose dynamic flavours
behind a
Cabal flag, which Hadrian will use whenever we are building a GHC
flavour that
requires dynamic libraries.

This is all part of a larger plan to build a dynamic stage 2 GHC by
default,
like with make, which in turn will fix a lot of test failures. See

Test Plan:
hadrian/build.sh _build/stage1/lib/package.conf.d/rts-1.0.conf
_build/stage1/lib/x86_64-.../ should contain many libHSrts-*.so

Reviewers: snowleopard, DavidEichmann, bgamari, erikd, simonmar

Reviewed By: snowleopard, DavidEichmann

Subscribers: rwbarton, carter

GHC Trac Issues: #15837

Differential Revision: https://phabricator.haskell.org/D5385

As outlined in #15925, hadrian bindists had not made a clear choice with
respect to relocatable GHCs and wrapper scripts. This commit implements
the policy described in the ticket. That is:

- the bindists ship {bin, lib} as they are, modulo the addition of
  haddock from stage2/bin
- we now _always_ generate wrapper scripts for all the programs that
  are in the bindist's bin/ directory

The idea being that anyone on Linux/Windows/OS X can just unpack
the binary distribution anywhere and start using bin/ghc, while the
installation process systematicaly generates wrapper scripts.

Test Plan: hadrian/build.sh binary-dist ; cd
_build/bindist/ghc-X.Y.Z-arch/; configure --prefix=/tmp/foo && make
install

Reviewers: snowleopard, bgamari, angerman

Reviewed By: snowleopard, bgamari, angerman

Subscribers: rwbarton, carter

GHC Trac Issues: #15925

Differential Revision: https://phabricator.haskell.org/D5371

This patch fixes a fairly long-standing bug (dating back to 2015) in
RdrName.bestImport, namely

   commit 9376249b
   Author: Simon Peyton Jones <simonpj@microsoft.com>
   Date:   Wed Oct 28 17:16:55 2015 +0000

   Fix unused-import stuff in a better way

In that patch got the sense of the comparison back to front, and
thereby failed to implement the unused-import rules described in
  Note [Choosing the best import declaration] in RdrName

This led to Trac #13064 and #15393

Fixing this bug revealed a bunch of unused imports in libraries;
the ones in the GHC repo are part of this commit.

The two important changes are

* Fix the bug in bestImport

* Modified the rules by adding (a) in
     Note [Choosing the best import declaration] in RdrName
  Reason: the previosu rules made Trac #5211 go bad again.  And
  the new rule (a) makes sense to me.

In unravalling this I also ended up doing a few other things

* Refactor RnNames.ImportDeclUsage to use a [GlobalRdrElt] for the
  things that are used, rather than [AvailInfo]. This is simpler
  and more direct.

* Rename greParentName to greParent_maybe, to follow GHC
  naming conventions

* Delete dead code RdrName.greUsedRdrName

Bumps a few submodules.

Reviewers: hvr, goldfire, bgamari, simonmar, jrtc27

Subscribers: rwbarton, carter

Differential Revision: https://phabricator.haskell.org/D5312

* Pass 'GHC/Prim.hs' to Haddock when processing 'ghc-prim'. This
  file is autogenerated for the sole purpose of giving Haddock
  something to process, so we really should make sure it gets
  through to Haddock!

* Add a "docs-haddock" build rule, which should build all Haddock docs
  that the Makefile builds by default (all libs + index for libs + ghc)

* Prune some unnecessary rules (esp. `gen_contents_index`)

Reviewers: bgamari, snowleopard

Reviewed By: snowleopard

Subscribers: alpmestan, snowleopard, rwbarton, carter

Differential Revision: https://phabricator.haskell.org/D5316

The path to the 'include' subdirectory of 'rts' includes a folder that
whose name is generated by Cabal and mentiones the architecture and OS.
For example:

    _build/stage1/lib/x86_64-osx-ghc-8.7.20181120/rts-1.0/include

Hadrian needs to be aware that Cabal renders architectures and OSes in
a slightly different way than GHC. There is already symmetric logic in
Cabal (for working with GHC environment files, which follow GHC's naming
conventions).

Test Plan: ./hadrian/build.sh -c "binary-dist" # on mac

Reviewers: snowleopard, alpmestan, bgamari

Reviewed By: snowleopard

Subscribers: rwbarton, carter

GHC Trac Issues: #15922

Differential Revision: https://phabricator.haskell.org/D5361

More generally, we so far assumed that the testsuite would be executed
with a flavour that's as comprehensive as perf in terms of available RTS
and library flavours (at least vanilla + dynamic + prof). This would
manifest itself concretely by needing 3 "ways" of the iserv program,
unconditionally.

We now only require the ways among vanilla, dynamic and prof that we
can find in our current Flavour's rtsWays.

Test Plan:
hadrian/build.sh --flavour={quick, quickest} test now goes through
(with a few failing tests, of course).

Reviewers: bgamari, tdammers

Reviewed By: tdammers

Subscribers: mpickering, RyanGlScott, rwbarton, carter

Differential Revision: https://phabricator.haskell.org/D5355

... in addition to ghc-iserv and ghc-iserv-prof, as it is required
to get 10+ tests to pass

Reviewers: bgamari, alpmestan

Reviewed By: alpmestan

Subscribers: alpmestan, rwbarton, carter

Differential Revision: https://phabricator.haskell.org/D5327

As it is required for 10+ tests.

Hadrian didn't give us a chance to build a given executable in vanilla
and profiling, simultaneously, under two different names. This patch
implements support for this in general and applies it to
ghc-iserv[-prof].

Test Plan: scc001 fails without this patch, passes with it.

Reviewers: snowleopard, bgamari

Reviewed By: bgamari

Subscribers: simonpj, ndmitchell, simonmar, rwbarton, carter

Differential Revision: https://phabricator.haskell.org/D5255

Sadly subtrees haven't worked quite as well as we would have liked for
developers. See Hadrian #440.