Alp Mestanogullari authored 5 years ago and Marge Bot committed 5 years ago

They are a particular type of perf tests. This patch introduces a
'stats_files_dir' configuration field in the testsuite driver where all
haddock timing files (and possibly others in the future) are assumed to live.
We also change both the Make and Hadrian build systems to pass respectively
$(TOP)/testsuite/tests/perf/haddock/ and
<build root>/stage1/haddock-timing-files/ as the value of that new
configuration field, and to generate the timing files in those directories
in the first place while generating documentation with haddock.

This new test type can be seen as one dedicated to examining stats files that
are generated while building a GHC distribution. This also lets us get rid of
the 'extra_files' directives in the all.T entries for haddock.base,
haddock.Cabal and haddock.compiler.

While windows and macOS are currently on case-insensitive file
systems, this poses no issue on those.  When cross compiling from
linux with a case sensitive file system and mingw providing only
lowercase headers, this in fact produces an issue.  As such we just
lowercase the import headers, which should still work fine on a
case insensitive file system and also enable mingw's headers to
be usable porperly.

Takenobu Tani authored 6 years ago and Marge Bot committed 6 years ago

This moves all URL references to Trac Wiki to their corresponding
GitLab counterparts.

This substitution is classified as follows:

1. Automated substitution using sed with Ben's mapping rule [1]
    Old: ghc.haskell.org/trac/ghc/wiki/XxxYyy...
    New: gitlab.haskell.org/ghc/ghc/wikis/xxx-yyy...

2. Manual substitution for URLs containing `#` index
    Old: ghc.haskell.org/trac/ghc/wiki/XxxYyy...#Zzz
    New: gitlab.haskell.org/ghc/ghc/wikis/xxx-yyy...#zzz

3. Manual substitution for strings starting with `Commentary`
    Old: Commentary/XxxYyy...
    New: commentary/xxx-yyy...

See also !539

[1]: https://gitlab.haskell.org/bgamari/gitlab-migration/blob/master/wiki-mapping.json

Ryan Scott authored 6 years ago and Marge Bot committed 6 years ago

This moves all URL references to Trac tickets to their corresponding
GitLab counterparts.

Ben Gamari authored 6 years ago and Marge Bot committed 6 years ago

The splitter is an evil Perl script that processes assembler code.
Its job can be done better by the linker's --gc-sections flag. GHC
passes this flag to the linker whenever -split-sections is passed on
the command line.

This is based on @DemiMarie's D2768.

Fixes Trac #11315
Fixes Trac #9832
Fixes Trac #8964
Fixes Trac #8685
Fixes Trac #8629

Alec Theriault authored 6 years ago and Ben Gamari committed 6 years ago

Summary:
This adds top-level configure flags '--with-intree-gmp' and
'--with-framework-preferred', both of which are especially relevant
on MacOS.

Besides gaining two new flags, Hadrian also had to be taught what to
do with the 'framework' in .cabal files.

Test Plan:
./boot && ./configure --with-intree-gmp && ./hadrian/build.sh
./boot && ./configure --with-gmp-framework-preferred && ./hadrian/build.sh # on macos

Reviewers: carter, snowleopard, alpmestan, hvr, goldfire, bgamari

Subscribers: rwbarton, erikd

GHC Trac Issues: #16001

Differential Revision: https://phabricator.haskell.org/D5417

Zejun Wu authored 6 years ago and Ben Gamari committed 6 years ago

Summary:
https://phabricator.haskell.org/D5169 built libghci for both vanilla way
and profiling way. We need to include both in the bindist list so they
will be installed.

Test Plan:
```
$ grep '^BuildFlavour' mk/build.mk
BuildFlavour=perf
$ make test_bindist
$ grep HSghc-prim bindist-list.uniq
ghc-8.7.20190101/libraries/ghc-prim/dist-install/build/HSghc-prim-0.5.3.o
ghc-8.7.20190101/libraries/ghc-prim/dist-install/build/HSghc-prim-0.5.3.p_o
ghc-8.7.20190101/libraries/ghc-prim/dist-install/build/libHSghc-prim-0.5.3.a
ghc-8.7.20190101/libraries/ghc-prim/dist-install/build/libHSghc-prim-0.5.3-ghc8.7.20190101.so
ghc-8.7.20190101/libraries/ghc-prim/dist-install/build/libHSghc-prim-0.5.3_p.a
```

Currently build logs from GitLab CI around around 7 megabytes each. Of this,
around 2 megabytes is latex output. This is quite silly as essentially all of
this output is unresolved references in the early latex iterations.
Here we silence this output.

However, to make sure that we don't silence errors we allow each xelatex
invocation besides the last to fail.

Zejun Wu authored 6 years ago and Ben Gamari committed 6 years ago

Summary:
We do this for vanilla way already.
Let's also merge sections for profiling way and use it instead of the .a
library when it exists.

Test Plan:
```
 $ inplace/bin/ghc-stage2 --interactive -prof -fexternal-interpreter
 GHCi, version 8.7.20180921: http://www.haskell.org/ghc/  :? for help
 Prelude> pid <- System.Posix.Process.getProcessID
 Prelude> maps <- readFile $ "/proc/" ++ show pid ++ "/maps"
 Prelude> pid
 3807346
 Prelude> putStrLn $ unlines $ take 20 $ lines maps
 00400000-02103000 r-xp 00000000 00:1a 199277344
 /data/users/watashi/ghc/inplace/lib/bin/ghc-iserv-prof
 02104000-02106000 r--p 01d03000 00:1a 199277344
 /data/users/watashi/ghc/inplace/lib/bin/ghc-iserv-prof
 02106000-02417000 rw-p 01d05000 00:1a 199277344
 /data/users/watashi/ghc/inplace/lib/bin/ghc-iserv-prof
 02417000-0280a000 rw-p 00000000 00:00 0
 [heap]
 40098000-400b0000 rwxp 000d2000 00:1a 199276023
 /data/users/watashi/ghc/libraries/bytestring/dist-install/build/HSbytestring-0.10.8.2.p_o
 400b7000-400d8000 rwxp 00000000 00:00 0
 401d1000-401d2000 rwxp 000e9000 00:1a 199276023
 /data/users/watashi/ghc/libraries/bytestring/dist-install/build/HSbytestring-0.10.8.2.p_o
 40415000-40419000 rwxp 0000b000 00:1a 199275165
 /data/users/watashi/ghc/libraries/deepseq/dist-install/build/HSdeepseq-1.4.4.0.p_o
 404f8000-40526000 rwxp 000af000 00:1a 199274234
 /data/users/watashi/ghc/libraries/ghc-prim/dist-install/build/HSghc-prim-0.5.3.p_o
```

Reviewers: simonmar, bgamari, austin, hvr

Reviewed By: simonmar

Subscribers: rwbarton, carter

Differential Revision: https://phabricator.haskell.org/D5169

Alec Theriault authored 6 years ago and Ben Gamari committed 6 years ago

Adds a `-fenable-ide-info` flag which instructs GHC to generate `.hie`
files (see the wiki page:
https://ghc.haskell.org/trac/ghc/wiki/HIEFiles).

This is a rebased version of Zubin Duggal's (@wz1000) GHC changes for
his GSOC project, as posted here:
https://gist.github.com/wz1000/5ed4ddd0d3e96d6bc75e095cef95363d.

Test Plan: ./validate

Reviewers: bgamari, gershomb, nomeata, alanz, sjakobi

Reviewed By: alanz, sjakobi

Subscribers: alanz, hvr, sjakobi, rwbarton, wz1000, carter

Differential Revision: https://phabricator.haskell.org/D5239

Moritz Angermann authored 7 years ago and Ben Gamari committed 7 years ago

Requires some ghc-cabal changes as well.

Test Plan: ./validate

Reviewers: hvr, bgamari

Reviewed By: bgamari

Subscribers: hsyl20, erikd, alpmestan, rwbarton, thomie, carter

Differential Revision: https://phabricator.haskell.org/D4453

Summary:
ld.gold is buggy when using -r and a linker script.  See upstream bug
https://sourceware.org/bugzilla/show_bug.cgi?id=22266

This has been causing various brokenness for the GHC runtime linker,
where we load these broken object files.

Test Plan: Test program from #14675

Reviewers: bgamari, RyanGlScott, alpmestan, hvr, erikd

Subscribers: rwbarton, thomie, erikd, carter

GHC Trac Issues: #14328, #14675, #14291

Differential Revision: https://phabricator.haskell.org/D4431

Ben Gamari authored 7 years ago and Ben Gamari committed 7 years ago

By request of @alexbiehl.
CCing @snowleopard and @alpmestan as the same should be done in Hadrian.

Bumps haddock submodule.

Reviewers: alexbiehl

Reviewed By: alexbiehl

Subscribers: rwbarton, thomie, carter, snowleopard, alpmestan, alexbiehl

Differential Revision: https://phabricator.haskell.org/D4365

Do not use $SHELL as $SHELL is the user's preferred interactive shell.
We do not want this to leak into the wrapper scripts.

This isn't needed anymore as we don't support GHC < 8 anymore.

This is a follow-up to 122f183d

Summary:
This tool can be used to generate dll's for any list of object files
given to it. It will then repartition them automatically to fit within
a dll and generates as many dll's as needed to do this. Cyclic dependencies
between these generated dlls are handle automatically so there is no need
to tell it how to partition.

It is also a lot more general than `dll-split` as it is able to split any
package not just `libGHC`. It also uses a trick using GNU style import libraries
to hide the splitting from the rest of the pipeline. Which means come linking time
you don't need to know which dll contains what symbol or how many split dlls were
created.

The import libraries are by default created with libtool. However since libtool is BFD
based it is very slow. So if present and detected by configure the `genlib` tool
from the msys2 project is used. This makes a difference of about ~45 minutes when compiling.

To install `genlib` run `pacman -Sy mingw-w64-$(uname -m)-tools-git`.

More detailed explaination of the process can be found here
https://ghc.haskell.org/trac/ghc/wiki/WindowsDynamicLinking

Test Plan: ./validate

Reviewers: austin, hvr, bgamari, erikd, simonmar

Reviewed By: bgamari

Subscribers: snowleopard, rwbarton, thomie, erikd, #ghc_windows_task_force

GHC Trac Issues: #5987

Differential Revision: https://phabricator.haskell.org/D3883

Tamar Christina authored 7 years ago and Ben Gamari committed 7 years ago

This patch removes dll-split from the code base, the reason is dll-split
no longer makes any sense. It was designed to split a dll in two, but we
now already have many more symbols than would fit inside two dlls. So we
need a third one. This means there's no point in having to maintain this
list as it'll never work anyway and the solution isn't scalable.

Test Plan: ./validate

Reviewers: austin, bgamari

Reviewed By: bgamari

Subscribers: rwbarton, thomie, #ghc_windows_task_force

GHC Trac Issues: #5987

Differential Revision: https://phabricator.haskell.org/D3882

Herbert Valerio Riedel authored 7 years ago and Ben Gamari committed 7 years ago

Cabal's parser has been rewritten in terms of Parsec (which is not
enabled yet in Cabal-2.0 by default, but can be enabled by a cabal
flag). The plan for Cabal is to drop support for the non-parsec parser,
so we need to prepare GHC to cope with new situation.

However, this means that lib:Cabal requires three new library
dependency submodules,

 - parsec
 - text
 - mtl

What complicates matters is that we need to build `ghc-cabal` early on
during the bootstrap phase which currently needs to invoke `ghc --make`
directly. So these additional dependencies need to be integrated into
the monolithic `ghc --make` invocation which produces the `ghc-cabal`
executable.

Test Plan: `./validate --fast` passed

Reviewers: austin, bgamari

Subscribers: erikd, phadej, rwbarton, thomie

Differential Revision: https://phabricator.haskell.org/D3757

Summary:
Initial implementation of split-section on Windows.

This also corrects section namings and uses the platform
convention of `$` instead of `.` to separate sections.

Implementation is based on @awson's patches to binutils.

Binutils requires some extra help when compiling the libraries
for GHCi usage. We drop the `-T` and use implicit scripts to amend
the linker scripts instead of replacing it.

Because of these very large GHCi object files, we need big-obj support,
which will be added by another patch.

Test Plan: ./validate

Reviewers: awson, austin, bgamari

Subscribers: dfeuer, rwbarton, thomie, snowleopard, #ghc_windows_task_force

GHC Trac Issues: #12913

Differential Revision: https://phabricator.haskell.org/D3383

Ben Gamari authored 7 years ago and Ben Gamari committed 7 years ago

terminfo no longer needs to be able to find the ncurses headers.
See https://github.com/judah/terminfo/pull/22.

Reviewers: austin, hvr

Subscribers: rwbarton, thomie, erikd

Differential Revision: https://phabricator.haskell.org/D3688

Douglas Wilson authored 7 years ago and Ben Gamari committed 7 years ago

This fixes the regressions in the haddock performance tests introduced
in c9eb4385.

Reviewers: austin, bgamari

Reviewed By: bgamari

Subscribers: rwbarton, thomie

GHC Trac Issues: #13789

Differential Revision: https://phabricator.haskell.org/D3629

Sometimes it's handy to change a compiler flag
for a library in stage{0,1,2}.

Usage example:

    libraries/binary_EXTRA_HC_OPTS += -O1
    libraries/containers_EXTRA_HC_OPTS += -O1
    libraries/bytestring_EXTRA_HC_OPTS += -O1

Here override default -O2 defined in .cabal files
for these libraries to speed build up.

Signed-off-by: Sergei Trofimovich <slyfox@gentoo.org>

The build failure when cross-compiling from
linux to windows looks like:

    HC [stage 1] ghc/stage2/build/tmp/ghc-stage2.exe
      Call hs_init_ghc() from your main() function to set these options.
  /usr/libexec/gcc/i686-w64-mingw32/ld: cannot find -lHSghc-8.3-0

Similar to commit 745032dd
("rts: tweak cross-compilation to mingw32") decision to split
stage2 should be done based based on TargetOS, not HostOS.

Signed-off-by: Sergei Trofimovich <slyfox@gentoo.org>

Simon Marlow authored 8 years ago and Ben Gamari committed 8 years ago

Since the introduction of -split-sections, using GHCi with the RTS
linker is really slow:

```
$ time (echo :quit | ./inplace/bin/ghc-stage2 --interactive -fexternal-interpreter)
GHCi, version 8.1.20170304: http://www.haskell.org/ghc/  :? for help
Prelude> Leaving GHCi.

real        0m3.793s
```

(when we use `-fexternal-interpreter` it uses the RTS linker by default,
you can make it use the system linker by adding `-dynamic`)

Building the GHCi libs doesn't take much time or space in the GHC build,
but makes things much quicker for people using the RTS linker:

```
$ time (echo :quit | ./inplace/bin/ghc-stage2 --interactive -fexternal-interpreter)
GHCi, version 8.1.20170304: http://www.haskell.org/ghc/  :? for help
Prelude> Leaving GHCi.

real        0m0.285s
```

So I propose that we build and ship them unconditionally.

Test Plan: validate, perf tests above

Reviewers: bgamari, austin, niteria, erikd, Phyx

Reviewed By: bgamari

Subscribers: rwbarton, thomie, snowleopard

Differential Revision: https://phabricator.haskell.org/D3298

rwbarton authored 8 years ago and Ben Gamari committed 8 years ago

After commit a50082c1 we use -ffunction-sections -fdata-sections
for all C compilations, when $1_$2_SplitSections is set. But that
variable was set in build-package.mk which is not run for the RTS.
As a result the RTS was not being split, leading to larger binaries.

This commit fixes RTS splitting by moving the definition of
$1_$2_SplitSections to distdir-opts.mk, which is run for the RTS
(and also from build-package.mk).

Test Plan:
manual ./validate and check that RTS and base .c files
are split, but not object files in the compiler

Reviewers: austin, bgamari

Reviewed By: bgamari

Subscribers: thomie, snowleopard, olsner

Differential Revision: https://phabricator.haskell.org/D3137

olsner authored 8 years ago and Ben Gamari committed 8 years ago

Previously this was added only to the RTS's C files (those are the bulk
of it though), but there are C bits in ghc-prim, integer-gmp and base
too.

Followup for #8405, allows the large table of character properties in
base to be stripped when not used.

Test Plan: validate

Reviewers: austin, bgamari, simonmar

Reviewed By: bgamari

Subscribers: thomie, snowleopard

Differential Revision: https://phabricator.haskell.org/D3121

Matthew Pickering authored 8 years ago and Ben Gamari committed 8 years ago

By default ghc-pkg is quite noisy as it can't find interface files.

Test Plan:
Look at the build log stderr and see if there are lots of "cannot find
any of" errors.

Reviewers: austin, bgamari

Reviewed By: bgamari

Subscribers: thomie, snowleopard

Differential Revision: https://phabricator.haskell.org/D3103

There is two types of options passed directly to 'ld'
(and not to 'gcc' driver):
    - CONF_LD_LINKER_OPTS_STAGE$4
    - EXTRA_LD_OPTS

This changedoes two things:

- split 'EXTRA_LD_OPTS' into two variables:
    - EXTRA_LD_OPTS (accepts 'gcc' wrapper options)
    - EXTRA_LD_LINKER_OPTS (accepts raw 'ld' options)

- wraps all LD_LINKER options as '-Wl,' when passed
  to 'gcc' driver.

Fixes https://phabricator.haskell.org/D2776



Signed-off-by: Sergei Trofimovich <siarheit@google.com>

This reverts f48f5a9e

The prefixing does not work as comma
is stripped by $(addprefix) macro:

The following call
    $$(addprefix -optl-Wl, $$($1_$2_$3_ALL_LD_OPTS))

prefixes options with "-optl-Wl" not with "-optl-Wl,"

The simplest breakage can be seen by adding
    SRC_LD_OPTS += -O1
to mk/build.mk:

  <no location info>: error:
    Warning: Couldn't figure out linker information!
             Make sure you're using GNU ld, GNU gold
             or the built in OS X linker, etc.
  gcc: error: unrecognized command line option '-Wl-O1'

Another problem with original change is loss of ability
to pass options to gcc as a linker driver, for example:
    SRC_LD_OPTS += -flto

Signed-off-by: Sergei Trofimovich <siarheit@google.com>

Ben Gamari authored 8 years ago and Ben Gamari committed 8 years ago

GHC uses gcc, not ld, for linking. Consequently all flags to be
interpreted by ld need to be prefixed by -optl,-Wl on the GHC command
line.

Test Plan: Validate on OpenBSD

Reviewers: austin, rwbarton

Reviewed By: rwbarton

Subscribers: thomie

Differential Revision: https://phabricator.haskell.org/D2776

dobenour authored 8 years ago and Ben Gamari committed 8 years ago

Test Plan: GHC CI

Reviewers: austin, bgamari

Reviewed By: bgamari

Subscribers: snowleopard, thomie

Maniphest Tasks: T74

Differential Revision: https://phabricator.haskell.org/D2732

Ben Gamari authored 8 years ago and Ben Gamari committed 8 years ago

This ensures that artifacts built with build-prog see these options.
Also spruce up comments.

Test Plan: Carefully read it.

Reviewers: austin, hvr, erikd

Reviewed By: erikd

Subscribers: thomie, erikd

Differential Revision: https://phabricator.haskell.org/D2673

Summary:
This omits -L and -l flags from the linker command line that shouldn't
be necessary because GHC will already add them via the -package-id
flags we pass.

This also reverts part of 90538d86
that rearranges the linker command line and causes some knock-on
problems (see D2618).

Test Plan: validate (need to validate on Windows too)

Reviewers: Phyx, bgamari, niteria, austin, erikd

Subscribers: thomie

Differential Revision: https://phabricator.haskell.org/D2639

GHC Trac Issues: #12738

Ben Gamari authored 8 years ago and Ben Gamari committed 8 years ago

Previously this was a relative path which worked in the GHC tree, but
failed elsewhere. This caused trouble for out-of-tree users as well as
Hadrian, which wants to move build artifacts out of the working
directory. Fixes #8040.

Test Plan: Validate

Reviewers: thomie, austin, snowleopard, hvr

Reviewed By: snowleopard, hvr

Subscribers: thomie

Differential Revision: https://phabricator.haskell.org/D2530

GHC Trac Issues: #8040

Markus Rothe authored 8 years ago and Ben Gamari committed 8 years ago

This avoids the interactive prompt of xelatex on error.

Specifically per-component macros and multiple libraries.

Contains Cabal submodule update.

Signed-off-by: Edward Z. Yang <ezyang@cs.stanford.edu>

Test Plan: validate

Reviewers: austin, bgamari

Reviewed By: austin, bgamari

Subscribers: hvr, thomie

Differential Revision: https://phabricator.haskell.org/D2059

The madness that is latex never ceases to amaze.