Alec Theriault authored 6 years ago and Ben Gamari committed 6 years ago

Summary:
This fixes some of the issues that surfaced when trying to build
dynamic GHC on OSX. Unfortunately, due some other `libffi`
issues, this doesn't completely fix dynamic builds on OSX.

  - Use 'extra-libraries' from .cabal files instead of hardcoding
    which packages need which extra libs. Also add support for
    'extra-lib-dirs'.

  - Make sure Hadrian looks in the right places to support both
    plain '<pkg>.buildinfo' and '<pkg>.buildinfo.in' files.

  - Make the '-rpath' support more robust across OS's (it previously
    didn't work on OSX and possibly windows either).

Reviewers: angerman, alpmestan, adamse, DavidEichmann, bgamari, Phyx

Subscribers: rwbarton, carter

GHC Trac Issues: #15990

Differential Revision: https://phabricator.haskell.org/D5409

Zejun Wu authored 6 years ago and Ben Gamari committed 6 years ago

Introduce ghci command wrapper, which can be used to cutomize ghci:
* process additionals actions before/after the command
* handle particular exceptions in given ways
* logging stats

We also split the timing and printing part of `timeIt` into different
functions.

Peter Trommler authored 6 years ago and Ben Gamari committed 6 years ago

There is only one place where UPDATE_SP was used. Instead of the
UPDATE_SP pseudo instruction build the list of instructions directly.

Alec Theriault authored 6 years ago and Ben Gamari committed 6 years ago

This means that we can query the package DB for haddock interfaces.
Haddock uses this in its testsuite. 'cabal {v1-,v2,}-haddock' also
uses this.

Also thread through to Haddock package-specific overrides of GHC
warnings.

Trac #16059 shows that when validity checking applications of type
synonyms, GHC sometimes wasn't checking the expanded type enough.
We must be careful, however, since checking both the expanded type as
well as the arguments to the type synonym can lead to exponential
blowup (see https://ghc.haskell.org/trac/ghc/ticket/16059#comment:4).
Nor can we omit checking either the expanded type or the argument for
correctness reasons.

The solution here is to introduce a new `ExpandMode` data type that
is plumbed through all of the type-validity-checking functions in
`TcValidity`. `ExpandMode` dictates whether we only check the
expanded type (`Expand`), only check the arguments (`NoExpand), or
both (`Both`). Importantly, if we check `Both` in the function for
validity checking type synonym applications, then we switch to
`NoExpand` when checking the arguments so as to avoid exponential
blowup. See `Note [Correctness and performance of type synonym validity
checking]` for the full story.

See Note [Cleanup on Windows].

Andrey Mokhov authored 6 years ago and Ben Gamari committed 6 years ago

See !100 (comment 2349).

Andrey Mokhov authored 6 years ago and Ben Gamari committed 6 years ago

* Update or remove a few outdated paragraphs
* Use consistent naming (e.g. "Stage1" instead of occasionally "stage1")
* Rewrite acknowledgements
* Some more minor revisions

Herbert Valerio Riedel authored 6 years ago and Ben Gamari committed 6 years ago

This also requires adapting `ghc-pkg` to use the new Cabal parsing API
as the old ReadP-based one has finally been evicted for good.

Hadrian bit finished by: Ben Gamari <ben@smart-cactus.org>

Previously we ACQUIRE_LOCK and RELEASE_LOCK but these compile to a noop in the
non-threaded RTS, as noted in #16150. Use OS_ACQUIRE_LOCK and OS_RELEASE_LOCK
instead.

When we are shutting down the pthread ticker we signal the start_cond condition
variable to ensure that the ticker thread wakes up and exits in a reasonable
amount of time. Previously, when the ticker thread would shut down it was
responsible for freeing the start_cond condition variable. However, this would
lead to a race wherein the ticker would free start_cond, then the main thread
would try to signal it in an effort to wake the ticker (#16150).

Avoid this by moving the mutex destruction to the main thread.

Instead of parsing and executing a statement or declaration directly we
now parse them first and then execute in a separate step. This gives us
the flexibility to inspect the parsed declaration before execution.
Using this we now inspect parsed declarations, and if it's a single
declaration of form `x = y` we execute it as `let x = y` instead, fixing
a ton of problems caused by poor declaration support in GHCi.

To avoid any users of the modules I left `execStmt` and `runDecls`
unchanged and added `execStmt'` and `runDecls'` which work on parsed
statements/declarations.

(See comments)

Because garbage collector calls `retainerProfile()` and `heapCensus()`,
GC times normally include some of PROF times too. To fix this we have
these lines:

    // heapCensus() is called by the GC, so RP and HC time are
    // included in the GC stats.  We therefore subtract them to
    // obtain the actual GC cpu time.
    stats.gc_cpu_ns      -=  prof_cpu;
    stats.gc_elapsed_ns  -=  prof_elapsed;

These variables are later used for calculating GC time excluding the
final GC (which should be attributed to EXIT).

    exit_gc_elapsed      = stats.gc_elapsed_ns - start_exit_gc_elapsed;

The problem is if we subtract PROF times from `gc_elapsed_ns` and then
subtract `start_exit_gc_elapsed` from the result, we end up subtracting
PROF times twice, because `start_exit_gc_elapsed` also includes PROF
times.

We now subtract PROF times from GC after the calculations for EXIT and
MUT times. The existing assertion that checks

    INIT + MUT + GC + EXIT = TOTAL

now holds. When we subtract PROF numbers from GC, and a new assertion

    INIT + MUT + GC + PROF + EXIT = TOTAL

also holds.

Fixes #15897. New assertions added in this commit also revealed #16102,
which is also fixed by this commit.

UNDERFLOW_FRAMEs don't have profiling headers so we have to use the
AP_STACK's function's CCS as the new frame's CCS.

Fixes one of the many bugs caught by concprog001 (#15508).

- Remove REGISTER_CC and REGISTER_CCS macros, add functions registerCC
  and registerCCS to Profiling.c.

- Reduce scope of symbols: CC_LIST, CCS_LIST, CC_ID, CCS_ID

- Document CC_LIST and CCS_LIST

[skip ci]

Summary:
This fixes #15471

In the typechecker we check that the splice has the right type but we
crucially don't zonk the generated expression. This is because we might
end up unifying type variables from outer scopes later on.

Reviewers: simonpj, goldfire, bgamari

Subscribers: rwbarton, carter

GHC Trac Issues: #15471

Differential Revision: https://phabricator.haskell.org/D5286

This helped me debug one of the bugs in #15508. I'm not sure if this is
a good idea, but it worked for me, so wanted to submit this as a MR.

This ticket enables the building of a `stage3` compiler by making the
build logic more consistent and predictable in Hadrian.

Two of the main changes are:

1. In order to build anything at stageN we use the package database
present at stageN. Fixing #16069
2. `haddock` and `ghc-tags` are built
as stage1 executables (with the stage1 compiler) rather than as
stage2 compiler. Fixing
[hadrian#661](https://github.com/snowleopard/hadrian/issues/661)

In order to build a stage3 compiler, you have to set the new `finalStage` hadrian option to `Stage3`.

Summary:
We currently permit any and all uses of unsaturated type
synonyms and type families in GHCi's `:kind` command, which allows
strange interactions like this one:

```
> :set -XTypeFamilies -XPolyKinds
> type family Id (a :: k)
> type instance Id a = a
> type Foo x = Maybe
> :kind! Id Foo
```

This is probably a stretch too far, so this patch moves to disallow
unsaturated synonyms that aren't at the top level (we still want to
allow `:kind Id`, for instance). We do this by augmenting `GhciCtxt`
with an additional `Bool` field to indicate if we are at the
outermost level of the type being passed to `:kind` or not. See
`Note [Unsaturated type synonyms in GHCi]` in `TcValidity` for the
full story.

Test Plan: make test TEST=T16013

Reviewers: goldfire, bgamari

Reviewed By: goldfire

Subscribers: simonpj, goldfire, rwbarton, carter

GHC Trac Issues: #16013

Differential Revision: https://phabricator.haskell.org/D5471

Summary:
We use 'gcc -B<base-location> --print-file-name mylib.a' as a way of
checking if 'gcc' can discover 'mylib.a' at the given location. However,
this can break down if there is a folder caller 'mylib.a' that 'gcc' can
discover. We can guard against this by explicitly checking that the path
returned by 'gcc' is a file.

This may seem like a far-fetched scenario, but since
3d17f1f1, we look for libraries without
any prefix or suffix (ie. 'extra-libraries: softfloat', we look for just
'softfloat' as well as 'softloat.a', 'softfloat.dll.a', etc.) which means
that there might actusally be a folder of that name in one of the base
locations.

Reviewers: Phyx, bgamari, hvr, angerman

Reviewed By: Phyx, angerman

Subscribers: angerman, rwbarton, carter

GHC Trac Issues: #16063

Differential Revision: https://phabricator.haskell.org/D5462

Debian Jessie only runs Python 3.4, whereas typing was introduced in Python 3.5.

This ensures that changes requiring haddock changes can be built under CI.

Zejun Wu authored 6 years ago and Ben Gamari committed 6 years ago

This is the hadrain version of {D5169}

* We build squashed .o and .p_o for ghci when `dynamicGhcPrograms` is
`False`
* We no longer build them for rts as ghci never loads it

we need https://github.com/haskell/cabal/pull/5592 for cabal to copy
the built `.p_o` file.

Test Plan:
```
$ grep dynamicGhc hadrian/UserSettings.hs
  , dynamicGhcPrograms = return False
$ touch ...
$ hadrian/build.sh --flavour=user -j --digest-or
$ find _build/stage1/libraries/ -name 'HS*-*.*o' | wc
     62      62    3664
```

```
$ grep -C3 dynamicGhc hadrian/UserSettings.hs
userFlavour :: Flavour
userFlavour = performanceFlavour
  { name = "user"
  , dynamicGhcPrograms = return False
  }
$ hadrian/build.sh -j --flavour=user test --verbose
Unexpected results from:
TEST="T3807 T9208 T9293 annth_make ghci057 haddock.Cabal haddock.base
haddock.compiler"

SUMMARY for test run started at Wed Dec  5 17:45:39 2018 PST
 0:03:16 spent to go through
    6708 total tests, which gave rise to
   26015 test cases, of which
   19290 were skipped

      29 had missing libraries
    6600 expected passes
      88 expected failures

       3 caused framework failures
       0 caused framework warnings
       1 unexpected passes
       7 unexpected failures
       0 unexpected stat failures
$ find _build -name 'HSbase*.*o'
_build/stage1/lib/x86_64-linux-ghc-8.7.20181204/base-4.12.0.0/HSbase-4.1
2.0.0.o
_build/stage1/lib/x86_64-linux-ghc-8.7.20181204/base-4.12.0.0/HSbase-4.1
2.0.0.p_o
_build/stage1/libraries/base/build/HSbase-4.12.0.0.o
_build/stage1/libraries/base/build/HSbase-4.12.0.0.p_o
```

Reviewers: bgamari, simonmar, snowleopard

Reviewed By: snowleopard

Subscribers: alpmestan, rwbarton, carter

GHC Trac Issues: #15779

Differential Revision: https://phabricator.haskell.org/D5270

Zejun Wu authored 6 years ago and Ben Gamari committed 6 years ago

Summary:
https://phabricator.haskell.org/D5169 built libghci for both vanilla way
and profiling way. We need to include both in the bindist list so they
will be installed.

Test Plan:
```
$ grep '^BuildFlavour' mk/build.mk
BuildFlavour=perf
$ make test_bindist
$ grep HSghc-prim bindist-list.uniq
ghc-8.7.20190101/libraries/ghc-prim/dist-install/build/HSghc-prim-0.5.3.o
ghc-8.7.20190101/libraries/ghc-prim/dist-install/build/HSghc-prim-0.5.3.p_o
ghc-8.7.20190101/libraries/ghc-prim/dist-install/build/libHSghc-prim-0.5.3.a
ghc-8.7.20190101/libraries/ghc-prim/dist-install/build/libHSghc-prim-0.5.3-ghc8.7.20190101.so
ghc-8.7.20190101/libraries/ghc-prim/dist-install/build/libHSghc-prim-0.5.3_p.a
```