Summary:
This patch implements Backpack for GHC.  It's a big patch but I've tried quite
hard to keep things, by-in-large, self-contained.

The user facing specification for Backpack can be found at:

    https://github.com/ezyang/ghc-proposals/blob/backpack/proposals/0000-backpack.rst

A guide to the implementation can be found at:

    https://github.com/ezyang/ghc-proposals/blob/backpack-impl/proposals/0000-backpack-impl.rst



Has a submodule update for Cabal, as well as a submodule update
for filepath to handle more strict checking of cabal-version.

Signed-off-by: Edward Z. Yang <ezyang@cs.stanford.edu>

Test Plan: validate

Reviewers: simonpj, austin, simonmar, bgamari, goldfire

Subscribers: thomie, mpickering

Differential Revision: https://phabricator.haskell.org/D1482

Signed-off-by: Edward Z. Yang <ezyang@cs.stanford.edu>

Signed-off-by: Edward Z. Yang <ezyang@cs.stanford.edu>

Signed-off-by: Edward Z. Yang <ezyang@cs.stanford.edu>

Signed-off-by: Edward Z. Yang <ezyang@cs.stanford.edu>

Martin Ceresa authored 8 years ago and Matthew Pickering committed 8 years ago

When looking up flags, we make sure to lookup the non-deprecated flags first
by ordering the list of flags.

Reviewers: bgamari, austin, mpickering

Reviewed By: mpickering

Subscribers: mpickering, thomie

Differential Revision: https://phabricator.haskell.org/D2541

GHC Trac Issues: #12574

This fixes the leak with `setProgArgv`. The problem was
that `setProgArgv` would not free the objects pointed
to by `prog_argc`, `prog_argv` when the globals were
changed resulting in a leak.

The only strictly necessary change is in `rts/RtsFlags.c`, but
the code in `System.Environment` was a bit confusing and not
exception safe, so I refactored it.

Test Plan: ./validate

Reviewers: simonmar, ezyang, austin, hvr, bgamari, erikd

Subscribers: thomie

Differential Revision: https://phabricator.haskell.org/D2576

GHC Trac Issues: #12664

Signed-off-by: Edward Z. Yang <ezyang@cs.stanford.edu>

Differential Revision: https://phabricator.haskell.org/D2574

these actually are complete, but due to the use of pattern guards, the
compiler does not see that. Refactor the code that it does.

Differential Revision: https://phabricator.haskell.org/D2574

Summary:
Keeping a promise I made to Simon to clean up these modules.

This change splits up the massive `TcDeriv` and `TcGenDeriv` modules into
somewhat more manageable pieces. The new modules are:

* `TcGenFunctor`: This contains the deriving machinery for `Functor`,
  `Foldable`, and `Traversable` (which all use the same underlying algorithm).
* `TcDerivInfer`: This is the new home for `inferConstraints`,
  `simplifyInstanceContexts`, and related functions, whose role is to come up
  with the derived instance context and subsequently simplify it.
* `TcDerivUtils`: This is a grab-bag module that contains several
  error-checking utilities originally in `TcDeriv`, as well as some functions
  that `TcDeriv` and `TcDerivInfer` both need.

The end result is that `TcDeriv` is now less than 1,600 SLOC (originally 2,686
SLOC), and `TcGenDeriv` is now about 2,000 SLOC (originally 2,964).

In addition, this also implements a couple of tiny refactorings:

* I transformed `type Condition = (DynFlags, TyCon) -> Validity` into
  `type Condition = DynFlags -> TyCon -> Validity`
* I killed the `DerivSpecGeneric` constructor for `DerivSpecMechanism`, and
  merged its functionality into `DerivSpecStock`. In addition,
  `hasStockDeriving` now contains key-value pairs for `Generic` and `Generic1`,
  so they're no longer treated as an awkward special case in `TcDeriv`.

Test Plan: ./validate

Reviewers: simonpj, austin, bgamari

Reviewed By: simonpj

Subscribers: thomie, mpickering

Differential Revision: https://phabricator.haskell.org/D2568

This was (accidentally?) introduced in 465c6c5d

See Note [Constraints and errors] in TcRnMonad.  This
patch fixes Trac #12124 in quite a neat way.

Comments only.

[ci skip]

It is a self-recursive function and hence a loop-breaker.

Signed-off-by: Sergei Trofimovich <siarheit@google.com>

Ben Gamari authored 8 years ago and Ben Gamari committed 8 years ago

This apparently should have been an import of rawSystem instead of
runProcess. Oops.

Fixes D2538.

Test Plan: Validate on Linux and Windows.

Reviewers: austin, snowleopard

Reviewed By: snowleopard

Subscribers: thomie

Differential Revision: https://phabricator.haskell.org/D2561

Summary:
Note that Cabal needs one more bugfix which is in PR to
fix GHC bootstrapping. But the rest of the patch is
ready for review.

Needs a filepath submodule update because cabal check
became more strict.

This patch handles the abstract-ification of Version and
PackageName.

Signed-off-by: Edward Z. Yang <ezyang@cs.stanford.edu>

Test Plan: validate

Reviewers: bgamari, austin

Subscribers: thomie

Differential Revision: https://phabricator.haskell.org/D2555

Somehow this testcase works on Darwin but not on Linux. This deserves
further investigation.

Ben Gamari authored 8 years ago and Ben Gamari committed 8 years ago

Test Plan: Validate

Reviewers: austin, snowleopard

Reviewed By: snowleopard

Subscribers: thomie

Differential Revision: https://phabricator.haskell.org/D2559

GHC Trac Issues: #8040.

The output contains the pid and executable path of the bash process
which spawned the failing process. It doesn't seem worth the effort to
cleanse this output; just ignore it.

It currently fails with,

    =====> T6132(normal) 1 of 1 [0, 0, 0]
    cd "./runghc/T6132.run" && "/Users/bgamari/ghc/inplace/test
    spaces/ghc-stage2" -c T6132.hs -dcore-lint -dcmm-lint
    -no-user-package-db -rtsopts -fno-warn-missed-specialisations
    -fshow-warning-groups -dno-debug-output
    Compile failed (exit code 1) errors were:

    T6132.hs:1:2: error: parse error on input ‘#!/’

    *** unexpected failure for T6132(normal)

Michael Snoyman authored 8 years ago and Ben Gamari committed 8 years ago

This means that, on POSIX systems, there will be only one ghc process
used for running scripts, as opposed to the current situation of a
runghc process and a ghc process. Beyond minor performance benefits of
not having an extra fork and resident process, the more important impact
of this is automatically getting proper signal handling. I noticed this
problem myself when running runghc as PID1 inside a Docker container.

I attempted to create a shim library for executeFile that would work for
both POSIX and Windows, but unfortunately I ran into issues with exit
codes being propagated correctly (see
https://github.com/fpco/replace-process/issues/2). Therefore, this patch
leaves the Windows behavior unchanged. Given that signals are a POSIX
issue, this isn't too bad a trade-off. If someone has suggestions for
better Windows _exec support, please let me know.

Reviewers: erikd, austin, bgamari

Reviewed By: bgamari

Subscribers: Phyx, thomie

Differential Revision: https://phabricator.haskell.org/D2538

Ben Gamari authored 8 years ago and Ben Gamari committed 8 years ago

This will allow us to split up Harbormaster output for the build and
test stages of validation.

Test Plan: `./validate --build-only && ./validate --testsuite-only`

Reviewers: thomie, hvr, austin

Differential Revision: https://phabricator.haskell.org/D2553

There was an awful leak where GHC permitted standalone `deriving`
declarations to create instances for unboxed sum or tuple types. This
fortifies the checks that GHC performs to catch this scenario and give
an appropriate error message.

Fixes #11509.

Test Plan: ./validate

Reviewers: goldfire, austin, bgamari

Reviewed By: bgamari

Subscribers: thomie

Differential Revision: https://phabricator.haskell.org/D2557

GHC Trac Issues: #11509

Currently, `Foreign.C.Types`, `Foreign.Ptr`, and `System.Posix.Types`
define `Read` and `Show` instances for the newtypes in those modules by
using `unsafeCoerce#`. We can clean up this hack by using the `newtype`
deriving strategy.

Reviewers: hvr, austin, bgamari

Reviewed By: bgamari

Subscribers: thomie

Differential Revision: https://phabricator.haskell.org/D2556

Nicolas Trangez authored 8 years ago and Ben Gamari committed 8 years ago

In GHC < 8.0.1, the value of `__GLASGOW_HASKELL_LLVM__`, exposed
through the preprocessor when compiled with `-fllvm`, was an integer
value, encoded according to some rules specified in the user guide.

Due to an oversight, in GHC 8.0.1 the value of this define became a
tuple, exposed as e.g. `(3, 7)`. This was an unintended regression.

This patch turns the value of the `__GLASGOW_HASKELL_LLVM__` definition
into a single integer again, but changes the formatting of said number
slightly. Before, any LLVM version where the major or minor component >=
10 would cause ambiguous values for `__GLASGOW_HASKELL_LLVM__`. With
this patch, the value is in line with `__GLASGOW_HASKELL__`, adding a
padding `0` in-between major and minor component if applicable (we
assume no minors >= 100 will ever exist).

The documentation in the user guide is updated accordingly, and a
reference is made in the 8.0.2 release notes.

Test Plan: validate

Reviewers: bgamari, erikd

Reviewed By: bgamari

Subscribers: thomie

Differential Revision: https://phabricator.haskell.org/D2552

GHC Trac Issues: #12628

Niklas Hambüchen authored 8 years ago and Ben Gamari committed 8 years ago

Reviewers: austin, bgamari

Reviewed By: bgamari

Subscribers: thomie

Differential Revision: https://phabricator.haskell.org/D2550

GHC Trac Issues: #12637

Adam Gundry authored 8 years ago and Ben Gamari committed 8 years ago

When DuplicateRecordFields is enabled, the mangling of selector names
was causing them to be reported as unused even if prefixed by an
underscore. This corrects the OccName used by the check.

Test Plan: New test overloadedrecflds/should_compile/T12609

Reviewers: austin, bgamari

Reviewed By: bgamari

Subscribers: thomie

Differential Revision: https://phabricator.haskell.org/D2549

GHC Trac Issues: #12609

Peter Trommler authored 8 years ago and Ben Gamari committed 8 years ago

Opcode lwa is a 64-bit opcode and allows a DS-form only.  This patch
generates lwa opcodes only when the offset is a multiple of 4.

Fixes #12621

Test Plan: validate

Reviewers: erikd, hvr, simonmar, austin, bgamari

Reviewed By: bgamari

Subscribers: thomie

Differential Revision: https://phabricator.haskell.org/D2547

GHC Trac Issues: #12621

Matthew Pickering authored 8 years ago and Ben Gamari committed 8 years ago

Previously the renamer assumed that *any* time we renamed a pattern, the
pattern was introducing new binders. This isn't true in pattern synonym
declarations where the pattern is used as part of a definition.

We add a special case to not warn in this situation.

Reviewers: simonpj, austin, bgamari

Reviewed By: simonpj

Subscribers: simonpj, thomie

Differential Revision: https://phabricator.haskell.org/D2545

GHC Trac Issues: #12615

Matthew Pickering authored 8 years ago and Ben Gamari committed 8 years ago

We were missing an appropiate *ConLike lookup in the case when
the pattern synonym was defined in a different module.

Reviewers: austin, bgamari, simonpj

Reviewed By: simonpj

Subscribers: simonpj, thomie

Differential Revision: https://phabricator.haskell.org/D2544

GHC Trac Issues: #11987

Summary:
NOTE: I have been able to do simple testing on emulated NUMA nodes.
           Real hardware would be needed for a proper test.

D2199 Added NUMA support for Linux, I have just filled in the missing pieces following
the description of the Linux APIs.

Test Plan:
Use `bcdedit.exe /set groupsize 2` to modify the kernel again (Similar to D2533).

This generates some NUMA nodes:

```
Logical Processor to NUMA Node Map:
NUMA Node 0:
**
--
NUMA Node 1:
--
**

Approximate Cross-NUMA Node Access Cost (relative to fastest):
     00  01
00: 1.1 1.1
01: 1.0 1.0
```

run ` ../test-numa.exe +RTS --numa -RTS`

and check PerfMon for NUMA allocations.

Reviewers: simonmar, erikd, bgamari, austin

Reviewed By: simonmar

Subscribers: thomie, #ghc_windows_task_force

Differential Revision: https://phabricator.haskell.org/D2534

GHC Trac Issues: #12602

Summary:
#8870 added as a temporary work around a much higher initial reserve
and committed stack space of 2mb. This is causing problems with other windows applications.

The hack was supposed to be temporary untill we could emit `__chkstk` instructions.
But GCC can emit stack checks automatically for us if `-fstack-check` is passed.

This will then emit calls to `___chkstk_ms` before stack allocations.

```
  633de0:       48 83 e0 f0             and    $0xfffffffffffffff0,%rax
  633de4:       e8 07 0c 0d 00          callq  7049f0 <___chkstk_ms>
  633de9:       48 29 c4                sub    %rax,%rsp
```

The hack is now no longer needed.

Test Plan: ./validate

Reviewers: austin, erikd, awson, bgamari

Reviewed By: bgamari

Subscribers: thomie, #ghc_windows_task_force

Differential Revision: https://phabricator.haskell.org/D2535

GHC Trac Issues: #12186

Ben Gamari authored 8 years ago and Ben Gamari committed 8 years ago

Test Plan: `validate --slow`

Reviewers: austin

Subscribers: thomie

Differential Revision: https://phabricator.haskell.org/D2536

GHC Trac Issues: #12019

Matthew Pickering authored 8 years ago and Ben Gamari committed 8 years ago

It is often convenient to copy the test invocation and remove this flag
in order to see compiler traces. Moving it to the end makes it easier to
remove.

Remove trailing whitespace

Reviewers: austin, thomie, bgamari

Reviewed By: bgamari

Differential Revision: https://phabricator.haskell.org/D2543

tmcgilchrist authored 8 years ago and Ben Gamari committed 8 years ago

The user guide says that we allow the user to use `specialise` or
`specialize` interchangeably but this wasn't the case for the relevant
flags. This patch adds aliases for the flags which control
specialisation.

Reviewers: erikd, austin, mpickering, bgamari

Reviewed By: mpickering, bgamari

Subscribers: mpickering, thomie

Differential Revision: https://phabricator.haskell.org/D2542

GHC Trac Issues: #12575

Tamar Christina authored 8 years ago and Ben Gamari committed 8 years ago

Windows support for more than 64 logical processors are implemented
using processor groups.

Essentially what it's doing is keeping the existing maximum of 64
processors and keeping the affinity mask a 64 bit value, but adds an
hierarchy above that.

This support was added to Windows 7 and so we need to at runtime detect
if the APIs are still there due to our minimum supported version being
Windows Vista.

The Maximum number of groups supported at this time is 4, so 256 logical
cores.  The group indices are 0 based. One thread can have affinity with
multiple groups.

See
https://msdn.microsoft.com/en-us/library/windows/desktop/ms684251.aspx
and particularly helpful is the whitepaper: 'Supporting Systems that
have more than 64 processors' at
https://msdn.microsoft.com/en-us/library/windows/hardware/dn653313.aspx

Processor groups are not guaranteed to be uniformly distributed nor
guaranteed to be filled before a next group is needed. The OS will
assign processors to groups based on physical proximity and will never
partially assign cores from one physical cpu to more than one group. If
one has two 48 core CPUs then you'd end up with two groups of 48 logical
cpus. Now add a 3rd CPU with 10 cores and the group it is assigned to
depends where the socket is on the board.

Test Plan:
./validate or make test -c . in the rts test folder.

This tests for regressions, to test this particular functionality
itself:

   <program> +RTS -N -qa -RTS

Test is detailed in description.

Reviewers: bgamari, simonmar, austin, erikd

Reviewed By: simonmar

Subscribers: thomie, #ghc_windows_task_force

Differential Revision: https://phabricator.haskell.org/D2533

GHC Trac Issues: #11054

The shadowed out bindings are accessible via qualified names like
Ghci1.foo.  Since they are accessable in the renamer the typechecker
should be able to see them too.  As a consequence they show up in :show
bindings.

This fixes T11547

Test Plan:
Fixed current tests to accomodate to new stuff in :show bindings
Added a test that verifies that the typechecker doesn't crash

Reviewers: austin, bgamari, simonpj

Reviewed By: simonpj

Subscribers: simonpj, thomie

Differential Revision: https://phabricator.haskell.org/D2447

GHC Trac Issues: #11547