GitLab

Tamar Christina authored Dec 25, 2019 and Ben Gamari committed Feb 05, 2020

GHC should make calls using process jobs when calling out to GCC and LD.
The reason is these use the exec () family of posix functions.  Window's
process model doesn't allow replacement of processes so this is emulated
by creating a new process and immediately exiting the old one.  Because
of this when using normal Windows wait functions you would return even
without the child process having finished.  In this case if you are
depending on data from the child you will enter a race condition.

The usual fix for this is to use process jobs and wait for the
termination of all children that have ever been spawn by the process you
called. But also waiting for the freeing of all resources.

Sylvain Henry authored Jan 15, 2020 and Marge Bot committed Jan 31, 2020

The package terminology is a bit of a mess. Cabal packages contain
components. Instances of these components when built with some
flags/options/dependencies are called units. Units are registered into
package databases and their metadata are called PackageConfig.

GHC only knows about package databases containing units. It is a sad
mismatch not fixed by this patch (we would have to rename parameters
such as `package-id <unit-id>` which would affect users).

This patch however fixes the following internal names:

- Renames PackageConfig into UnitInfo.
- Rename systemPackageConfig into globalPackageDatabase[Path]
- Rename PkgConfXX into PkgDbXX
- Rename pkgIdMap into unitIdMap
- Rename ModuleToPkgDbAll into ModuleNameProvidersMap
- Rename lookupPackage into lookupUnit
- Add comments on DynFlags package related fields

It also introduces a new `PackageDatabase` datatype instead of
explicitly passing the following tuple: `(FilePath,[PackageConfig])`.

The `pkgDatabase` field in `DynFlags` now contains the unit info for
each unit of each package database exactly as they have been read from
disk. Previously the command-line flag `-distrust-all-packages` would
modify these unit info. Now this flag only affects the "dynamic"
consolidated package state found in `pkgState` field. It makes sense
because `initPackages` could be called first with this
`distrust-all-packages` flag set and then again (using ghc-api) without
and it should work (package databases are not read again from disk when
`initPackages` is called the second time).

Bump haddock submodule

Ryan Scott authored Nov 13, 2019 and Marge Bot committed Nov 23, 2019

This reverts a part of commit
7bc5d6c6 that causes all arguments
to `-optc` (and `-optcxx`) to be passed twice to the C/C++ compiler,
once in reverse order and then again in the correct order. While
passing duplicate arguments is usually harmless it can cause breakage
in this pattern, which is employed by Hackage libraries in the wild:

```
ghc Foo.hs foo.c -optc-D -optcFOO
```

As `FOO -D -D FOO` will cause compilers to error.

Fixes #17471.

Ben Gamari authored Oct 01, 2019 and Marge Bot committed Nov 04, 2019

Issue #1110 was apparently due to a bug in Vista which prevented GCC
from finding its binaries unless we explicitly added it to PATH.
However, this workaround was incorrectly applied on non-Windows
platforms as well, resulting in ill-formed PATHs (#17266).

Fixes #17266.

Andreas Klebinger authored Oct 20, 2019 and Marge Bot committed Oct 23, 2019

19 times out of 20 we already have dynflags in scope.

We could just always use `return dflags`. But this is in fact not free.
When looking at some STG code I noticed that we always allocate a
closure for this expression in the heap. Clearly a waste in these cases.

For the other cases we can either just modify the callsite to
get dynflags or use the _D variants of withTiming I added which
will use getDynFlags under the hood.

 - Remove unneeded ones

 - Use <..> for inter-package.
   Besides general clean up, helps distinguish between the RTS we link
   against vs the RTS we compile for.

John Ericson authored Jul 12, 2019 and Marge Bot committed Oct 04, 2019

not found.

John Ericson authored Jul 12, 2019 and Marge Bot committed Oct 04, 2019

This has two benefits:

1. One less hunk of code dependent on DynFlags

2. Add a little bit of error granularity to distrinugish between missing
   data and bad data. This could someday be shared with ghc-pkg which
   aims to work even with a missing file. I also am about to to make
   --supported-extensions use this too.

Niklas Hambüchen authored Aug 18, 2019 and Marge Bot committed Sep 11, 2019

Until now, giving `-optl` linker flags to `ghc` on the command line placed
them in the wrong place in the `ld` command line:

They were given before all the Haskell libararies, when they should appear after.

Background:
Most linkers like `ld.bfd` and `ld.gold`, but not the newer LLVM `lld`, work in
a way where the order of `-l` flags given matters; earlier `-lmylib1` flags are
supposed to create "holes" for linker symbols that are to be filled with later
`lmylib2` flags that "fill the holes" for these symbols.

As discovered in
https://github.com/haskell/cabal/pull/5451#issuecomment-518001240,
the `-optl` flags appeared before e.g. the

    -lHStext-1.2.3.1
    -lHSbinary-0.8.6.0
    -lHScontainers-0.6.0.1

flags that GHC added at the very end.

Haskell libraries typically depend on C libraries, so `-lHS*` flags will create
holes for the C libraries to fill in, but that only works when those libraries'
`-l` flags are given **after** the `-lHS*` flags; until now they were given
before, which was wrong.

This meant that Cabal's `--ld-options` flag and `ld-options` `.cabal` file field
were pretty ineffective, unless you used the `--ld-option=--start-group` hack as
(https://github.com/haskell/cabal/pull/5451#issuecomment-406761676) that
convinces the classical linkers to not be dependent on the order of linker flags
given.

This commit fixes the problem by simply flipping the order, putting `-optl`
flags at the end, after Haskell libraries.

The code change is effectively only `args1 ++ args` -> `args ++ args1`
but the commit also renames the variables for improved clarity.

Simple way to test it:

    ghc --make Main.hs -fforce-recomp -v -optl-s

on a `Main.hs` like:

    import qualified Data.Set as Set
    main = print $ Set.fromList "hello"

Ryan Scott authored Sep 07, 2019 and Marge Bot committed Sep 09, 2019

`SysTools.Terminal.queryCygwinTerminal` now exists in the `Win32`
library under the name `isMinTTYHandle` since `Win32-2.5.0.0`.
(GHC 8.4.4 ships with `Win32-2.6.1.0`, so this is well within GHC's
support window.) We can therefore get replace `queryCygwinTerminal`
with `isMinTTYHandle` and delete quite a bit of code from
`SysTools.Terminal` in the process.

Along the way I needed to replace some uses of `#if defined x` with
`#if defined(x)` to please the CI linters.

Ömer Sinan Ağacan authored Aug 23, 2019 and Marge Bot committed Aug 29, 2019

There were two problems with LLVM version checking:

- The parser would only parse x and x.y formatted versions. E.g. 1.2.3
  would be rejected.

- The version check was too strict and would reject x.y formatted
  versions. E.g. when we support version 7 it'd reject 7.0 ("LLVM
  version 7.0") and only accept 7 ("LLVM version 7").

We now parse versions with arbitrarily deep minor numbering (x.y.z.t...)
and accept versions as long as the major version matches the supported
version (e.g. 7.1, 7.1.2, 7.1.2.3 ...).

Alp Mestanogullari authored Jul 11, 2019 and Marge Bot committed Jul 13, 2019

This patch was motivated by some performance characterization work done
for #16822, where we suspected that GHC was spending a lot of time waiting
on the linker to be done. (That turned out to be true.)

The tracing is taken care of by ErrUtils.withTiming, so this patch just defines
and uses a little wrapper around that function in all the helpers for
calling the various systools (C compiler, linker, unlit, ...).

With this patch, assuming a GHC executable linked against an eventlog-capable
RTS (RTS ways that contain the debug, profiling or eventlog way units), we can
measure how much time is spent in each of the SysTools when building hello.hs
by simply doing:

  ghc hello.hs -ddump-timings +RTS -l

The event names are "systool:{cc, linker, as, unlit, ...}".

Erik de Castro Lopo authored Jun 22, 2019 and Marge Bot committed Jun 24, 2019

LLVM version numberinf changed recently. Previously, releases were numbered
4.0, 5.0 and 6.0 but with version 7, they dropped the redundant ".0".

Fix requires for Llvm detection and some code.

John Ericson authored May 31, 2019 and Marge Bot committed Jun 19, 2019

This matches GHC itself getting the target platform from there.

John Ericson authored May 30, 2019 and Marge Bot committed Jun 19, 2019

ghc-pkg needs to be aware of platforms so it can figure out which
subdire within the user package db to use. This is admittedly
roundabout, but maybe Cabal could use the same notion of a platform as
GHC to good affect too.

Ben Gamari authored Jun 03, 2019 and Marge Bot committed Jun 13, 2019

Previously we would pass flags intended for the C compiler to the C++
compiler (see #16738). This would cause, for instance, `-std=gnu99` to
be passed to the C++ compiler, causing spurious test failures. Fix this
by maintaining a separate set of flags for C++ compilation invocations.

John Ericson authored May 30, 2019 and Marge Bot committed Jun 07, 2019

ghc-pkg and ghc already both needed this. I figure it is better to
deduplicate, especially seeing that changes to one (FreeBSD CPP) didn't
make it to the other.

Additionally in !1090 I make ghc-pkg look up the settings file, which
makes it use the top dir a bit more widely. If that lands, any
difference in the way they find the top dir would be more noticable.

That change also means sharing more code between ghc and ghc-package
(namely the settings file parsing code), so I'd think it better to get
off the slipperly slope of duplicating code now.

Ryan Scott authored Mar 12, 2019 and Marge Bot committed Mar 15, 2019

This moves all URL references to Trac tickets to their corresponding
GitLab counterparts.

Ben Gamari authored Nov 26, 2018 and Marge Bot committed Mar 05, 2019

The splitter is an evil Perl script that processes assembler code.
Its job can be done better by the linker's --gc-sections flag. GHC
passes this flag to the linker whenever -split-sections is passed on
the command line.

This is based on @DemiMarie's D2768.

Fixes Trac #11315
Fixes Trac #9832
Fixes Trac #8964
Fixes Trac #8685
Fixes Trac #8629

Simon Marlow authored Dec 07, 2018 and Ben Gamari committed Dec 07, 2018

Test Plan: validate using LLD as the linker (TODO)

Reviewers: bgamari, angerman, kavon, erikd

Reviewed By: bgamari

Subscribers: watashi, rwbarton, carter

Differential Revision: https://phabricator.haskell.org/D5336

Roland Senn authored Nov 22, 2018 and Ben Gamari committed Nov 22, 2018

Test Plan: make test TEST=T14452

Reviewers: hvr, bgamari, monoidal, thomie, osa1

Reviewed By: osa1

Subscribers: rwbarton, carter

GHC Trac Issues: #14452

Differential Revision: https://phabricator.haskell.org/D5318

Summary: I'm currently trying to make `hadrian` work as a build system
on FreeBSD (https://ghc.haskell.org/trac/ghc/ticket/15860).
I'm still having some issues with `libgmp` but one can get a working
`ghc` using `--integer-simple` and this patch.

Reviewers: bgamari, erikd, alpmestan

Reviewed By: alpmestan

Subscribers: rwbarton, carter

Differential Revision: https://phabricator.haskell.org/D5335

Summary:
GHC 8.6.1 is out, so now GHC's support window only extends
back to GHC 8.4. This means we can delete gobs of code that were
only used for GHC 8.2 support. Hooray!

Test Plan: ./validate

Reviewers: bgamari, Phyx, erikd

Reviewed By: bgamari, Phyx

Subscribers: rwbarton, erikd, carter

Differential Revision: https://phabricator.haskell.org/D5192

Simon Marlow authored Sep 28, 2018 and Krzysztof Gogolewski committed Sep 28, 2018

Summary:
I noticed while playing around with
https://github.com/fbsamples/ghc-hotswap/ that the main binary needs to
have a custom main function to set `config.keep_cafs = true` when
initialising the runtime. This is pretty annoying, it means an extra
C file with some cryptic incantations in it, and a `-no-hs-main` flag.

So I've replaced this with a link-time flag to GHC, `-fkeep-cafs` that
does the same thing.

Test Plan:
New unit test that tests for the RTS's GC'd CAFs assertion, and also
the -keep-cafs flag.

Reviewers: bgamari, osa1, erikd, noamz

Reviewed By: osa1

Subscribers: rwbarton, carter

Differential Revision: https://phabricator.haskell.org/D5183

Summary:
This completes the work started in D4227 by using just 'getExecutablePath'
in ghc and ghc-pkg when building with base >= 4.11.0.

On the long term, we will be able to simply kill the existing code that
follows (or not) symlinks and just get this behaviour for free from
getExecutable. For now we however have to require base >= 4.11.0 to be able
to just use getExecutablePath under Windows, and use the current code when
building with an older base.

Original code by @alpmestan commandeering since patch has been stale
and bug remains open.

Test Plan: Validate

Reviewers: angerman, bgamari, erikd, alpmestan

Reviewed By: bgamari

Subscribers: carter, rwbarton, thomie

GHC Trac Issues: #14483

Differential Revision: https://phabricator.haskell.org/D4229

Michael Sloan authored Jul 26, 2018 and Ben Gamari committed Jul 27, 2018

This change was previously part of
[D4904](https://phabricator.haskell.org/D4904), but is being split off
to aid in getting this reviewed and merged.

* The compiler code is built with `NoImplicitPrelude`, but GHCi's
  modules are incompatible with it. So, this adds the pragma to all GHCi
  modules that didn't have it, and adds imports of Prelude.

* In order to run GHC within itself, a `call of 'initGCStatistics`
  needed to be skipped. This uses CPP to skip it when
  `-DGHC_LOADED_INTO_GHCI` is set.

* There is an environment variable workaround suggested by Ben Gamari
  [1], where `_GHC_TOP_DIR` can be used to specify GHC's top dir if `-B`
  isn't provided.  This can be used to solve a problem where the GHC being
  run within GHCi attempts to look in `inplace/lib/lib/` instead of
  `inplace/lib/`.

[1]: https://phabricator.haskell.org/D4904#135438

Reviewers: goldfire, bgamari, erikd, alpmestan

Reviewed By: alpmestan

Subscribers: alpmestan, lelf, rwbarto...

Alp Mestanogullari authored Apr 17, 2018 and Tamar Christina committed Apr 17, 2018

Summary:
This patch affects several files that affect how we detect mingw and perl
on Windows. The initial motivation is:

    https://github.com/snowleopard/hadrian/issues/564

where, with Hadrian building relocatable (non-inplace) GHCs, the current
detection mechanism falls short by e.g only trying $topdir/../mingw. But
in Hadrian, for reasons given in that issue, we would need to store e.g mingw
under $topdir/../../mingw except for binary distributions, where we want
to follow the existing structure, in which case $topdir/../mingw is correct. So
we need to support both, which is what this patch hopefully implements.

Test Plan: ./validate

Reviewers: Phyx, hvr, bgamari, erikd

Reviewed By: Phyx

Subscribers: snowleopard, thomie, carter

Differential Revision: https://phabricator.haskell.org/D4598

Oleg Grenrus authored Mar 25, 2018 and Ben Gamari committed Mar 25, 2018

LLVM, like GHC, processes flags in the order that they appear.
Consequently, we need to ensure the user-provided flags appear last so
they can override flags produced by GHC. See #14821.

Test Plan: `ghc -O2 -optlo-O2 -v3 $FILE` and ensure that `opt` and `llc`
are invoked with `-O2`.

Subscribers: rwbarton, thomie, carter

GHC Trac Issues: #14821

Differential Revision: https://phabricator.haskell.org/D4421

Ryan Scott authored Mar 13, 2018 and David Feuer committed Mar 13, 2018

GHC 8.4.1 is out, so now GHC's support window only extends
back to GHC 8.2. This means we can delete gobs of code that were
only used for GHC 8.0 support. Hooray!

Test Plan: ./validate

Reviewers: bgamari, erikd, dfeuer

Reviewed By: bgamari, dfeuer

Subscribers: alexbiehl, dfeuer, rwbarton, thomie, carter

Differential Revision: https://phabricator.haskell.org/D4492

I noticed while trying to test against LLVM 5.0 that GHC would throw "Couldn't
figure out linker information" warnings due to LLD being chosen by configure.
Adding detection support to silence these is simple enough, let's just do it.

Andreas Klebinger authored Jan 15, 2018 and Ben Gamari committed Jan 15, 2018

On Windows GHC enforces currently that the real executable is named
ghc.exe/ghc-stage[123].exe.

I don't see a good reason why this is neccessary.
This patch removes this restriction and fixes #14652

Test Plan: ci

Reviewers: bgamari, Phyx

Reviewed By: Phyx

Subscribers: Phyx, rwbarton, thomie, carter

GHC Trac Issues: #14652

Differential Revision: https://phabricator.haskell.org/D4296

Ben Gamari authored Nov 23, 2017 and Ben Gamari committed Nov 23, 2017

Test Plan: Validate on Linux and Windows

Reviewers: erikd

Subscribers: rwbarton, thomie

Differential Revision: https://phabricator.haskell.org/D4225

Summary:
SysTools and DriverTools have an annoying mutual dependency.
They also each contain pieces of the linker. In order for
changes to be shared between the library and the exe linking
code this dependency needs to be broken in order to avoid
using hs-boot files.

Reviewers: austin, bgamari, erikd

Reviewed By: bgamari

Subscribers: rwbarton, thomie

Differential Revision: https://phabricator.haskell.org/D4071

Herbert Valerio Riedel authored Sep 19, 2017 and Ben Gamari committed Sep 19, 2017

This switches the compiler/ component to get compiled with
-XNoImplicitPrelude and a `import GhcPrelude` is inserted in all
modules.

This is motivated by the upcoming "Prelude" re-export of
`Semigroup((<>))` which would cause lots of name clashes in every
modulewhich imports also `Outputable`

Reviewers: austin, goldfire, bgamari, alanz, simonmar

Reviewed By: bgamari

Subscribers: goldfire, rwbarton, thomie, mpickering, bgamari

Differential Revision: https://phabricator.haskell.org/D3989

Rufflewind authored Dec 17, 2016 and Ben Gamari committed Dec 17, 2016

The previous detection mechanism allowed environment variables (ANSICON,
ConEmuANSI, TERM) to supersede the fact that the stderr is not a
terminal, which is probably what led to color codes appearing in the
stderr of the tests (see: 847d2293).

This commit changes the detection mechanism to detect Cygwin/MSYS2
terminals in a more reliable manner, avoiding the use of environment
variables entirely.

Test Plan: validate

Reviewers: Phyx, austin, erikd, bgamari

Reviewed By: Phyx, bgamari

Subscribers: RyanGlScott, thomie

Differential Revision: https://phabricator.haskell.org/D2809