GitLab

Summary:
Add prettyprinter tests, which take a file, parse it, pretty print it,
re-parse the pretty printed version and then compare the original and
new ASTs (ignoring locations)

Updates haddock submodule to match the AST changes.

There are three issues outstanding

1. Extra parens around a context are not reproduced. This will require an
   AST change and will be done in a separate patch.

2. Currently if an `HsTickPragma` is found, this is not pretty-printed,
   to prevent noise in the output.

   I am not sure what the desired behaviour in this case is, so have left
   it as before. Test Ppr047 is marked as expected fail for this.

3. Apart from in a context, the ParsedSource AST keeps all the parens from
   the original source.  Something is happening in the renamer to remove the
   parens around visible type application, causing T12530 to fail, as the
   dumped splice decl is after the renamer.

   This needs to be fixed by keeping the parens, but I do not know where they
   are being removed.  I have amended the test to pass, by removing the parens
   in the expected output.

Test Plan: ./validate

Reviewers: goldfire, mpickering, simonpj, bgamari, austin

Reviewed By: simonpj, bgamari

Subscribers: simonpj, goldfire, thomie, mpickering

Differential Revision: https://phabricator.haskell.org/D2752

GHC Trac Issues: #3384

This patch replaces calls to barf() in loadArchive() with proper
error handling.

Test Plan: GHC CI

Reviewers: rwbarton, erikd, hvr, austin, simonmar, bgamari

Reviewed By: bgamari

Subscribers: thomie

Tags: #ghc

Differential Revision: https://phabricator.haskell.org/D2652

GHC Trac Issues: #12388

We want to avoid using SIGALRM whenever possible since we will interrupt
long-running system calls. See #10840.

Test Plan: Validate on Darwin

Reviewers: austin, erikd, simonmar

Reviewed By: simonmar

Subscribers: thomie

Differential Revision: https://phabricator.haskell.org/D2796

GHC Trac Issues: #10840

See comments.

Static string optimisation means we get a ForeignPtr with an IORef
inside it, leading to a different error.

Summary:
This commit makes various improvements and addresses some issues with
Compact Regions (aka Compact Normal Forms).

This was the most important thing I wanted to fix.  Compaction
previously prevented GC from running until it was complete, which
would be a problem in a multicore setting.  Now, we compact using a
hand-written Cmm routine that can be interrupted at any point.  When a
GC is triggered during a sharing-enabled compaction, the GC has to
traverse and update the hash table, so this hash table is now stored
in the StgCompactNFData object.

Previously, compaction consisted of a deepseq using the NFData class,
followed by a traversal in C code to copy the data.  This is now done
in a single pass with hand-written Cmm (see rts/Compact.cmm). We no
longer use the NFData instances, instead the Cmm routine evaluates
components directly as it compacts.

The new compaction is about 50% faster than the old one with no
sharing, and a little faster on average with sharing (the cost of the
hash table dominates when we're doing sharing).

Static objects that don't (transitively) refer to any CAFs don't need
to be copied into the compact region.  In particular this means we
often avoid copying Char values and small Int values, because these
are static closures in the runtime.

Each Compact# object can support a single compactAdd# operation at any
given time, so the Data.Compact library now enforces mutual exclusion
using an MVar stored in the Compact object.

We now get exceptions rather than killing everything with a barf()
when we encounter an object that cannot be compacted (a function, or a
mutable object).  We now also detect pinned objects, which can't be
compacted either.

The Data.Compact API has been refactored and cleaned up.  A new
compactSize operation returns the size (in bytes) of the compact
object.

Most of the documentation is in the Haddock docs for the compact
library, which I've expanded and improved here.

Various comments in the code have been improved, especially the main
Note [Compact Normal Forms] in rts/sm/CNF.c.

I've added a few tests, and expanded a few of the tests that were
there.  We now also run the tests with GHCi, and in a new test way
that enables sanity checking (+RTS -DS).

There's a benchmark in libraries/compact/tests/compact_bench.hs for
measuring compaction speed and comparing sharing vs. no sharing.

The field totalDataW in StgCompactNFData was unnecessary.

Test Plan:
* new unit tests
* validate
* tested manually that we can compact Data.Aeson data

Reviewers: gcampax, bgamari, ezyang, austin, niteria, hvr, erikd

Subscribers: thomie, simonpj

Differential Revision: https://phabricator.haskell.org/D2751

GHC Trac Issues: #12455

I failed at the last attempt.

Test Plan: make test TEST=T12919

Reviewers: bgamari, austin

Reviewed By: bgamari

Subscribers: thomie

Differential Revision: https://phabricator.haskell.org/D2788

GHC Trac Issues: #12919

Removed the alignment for strings and mark then as cstring sections in
the generated asm so the linker can merge duplicate sections.

Reviewers: rwbarton, trofi, austin, trommler, simonmar, hvr, bgamari

Reviewed By: hvr, bgamari

Subscribers: simonpj, hvr, thomie

Differential Revision: https://phabricator.haskell.org/D1290

GHC Trac Issues: #9577

This reverts commit 19ae1423.

Summary:
This fixes tests rn017, T7672 and closed #12930.

Both these tests were self referential module imports through hs-boot
files. As a result, I am quite suspicious of what the ImpAll constructor is
used for. I had a brief hunt around but couldn't immediately see whether
it was necessary.

Reviewers: austin, bgamari

Subscribers: simonpj, thomie, nomeata

Differential Revision: https://phabricator.haskell.org/D2793

GHC Trac Issues: #12930

Summary:
Visible API changes:

* The C struct `GCDetails` gives the stats about a single GC.  This is
  passed to the `gcDone()` callback if one is set via the
  RtsConfig. (previously we just passed a collection of values, so this
  is more extensible, at the expense of breaking the existing API)

* `RTSStats` gives cumulative stats since the start of the program,
  and includes the `GCDetails` for the most recent GC.  This struct
  can be obtained via `getRTSStats()` (the old `getGCStats()` has been
  removed, and `getGCStatsEnabled()` has been renamed to
  `getRTSStatsEnabled()`)

Improvements:

* The per-GC stats and cumulative stats are now cleanly separated.

* Inside the RTS we have a top-level `RTSStats` struct to keep all our
  stats in, previously this was just a collection of strangely-named
  variables.  This struct is mostly just copied in `getRTSStats()`, so
  the implementation of that function is a lot shorter.

* Types are more consistent.  We use a uint64_t byte count for all
  memory values, and Time for all time values.

* Names are more consistent.  We use a suffix `_bytes` for all byte
  counts and `_ns` for all time values.

* We now collect information about the amount of memory in large
  objects and compact objects in `GCDetails`. (the latter was the reason
  I started doing this patch but it seems to have ballooned a bit!)

* I fixed a bug in the calculation of the elapsed MUT time, and added
  an ASSERT to stop the calculations going wrong in the future.

For now I kept the Haskell API in `GHC.Stats` the same, by
impedence-matching with the new API.  We could either break that API
and make it match the C API more closely, or we could add a new API
and deprecate the old one.  Opinions welcome.

This stuff is very easy to get wrong, and it's hard to test.  Reviews
welcome!

Test Plan:
manual testing
validate

Reviewers: bgamari, niteria, austin, ezyang, hvr, erikd, rwbarton, Phyx

Subscribers: thomie

Differential Revision: https://phabricator.haskell.org/D2756

Summary:
We mentioned that there were "some programs" that failed to typecheck
due to #12784, but given how surprisingly common this issue has been, it'd
be prudent to at least give one example of the bug in the release notes.

Reviewers: simonpj, bgamari, austin, rwbarton

Reviewed By: rwbarton

Subscribers: thomie

Differential Revision: https://phabricator.haskell.org/D2786

GHC Trac Issues: #12784

Summary:
Fix issues preventing x86 GHC to build on Windows and
fix segfault in the testsuite.

Test Plan: ./validate

Reviewers: austin, erikd, simonmar, bgamari

Reviewed By: bgamari

Subscribers: #ghc_windows_task_force, thomie

Differential Revision: https://phabricator.haskell.org/D2789

Summary:
D2716 introduced colors into the output of GHC.
These color ourputs are done using escape characters output
to the terminal.

These however are wreaking havoc on the testsuite output as now
no stderr with a warning or error will match anymore.

Instead of accepting the new codes as expected values instead I
turn them off. So the testsuite is consistent on platforms/terminals we
don't support colors on.

Test Plan:
any test that outputs colors. e.g.

make test TEST=T9576

Reviewers: austin, Rufflewind, bgamari

Subscribers: thomie, #ghc_windows_task_force

Differential Revision: https://phabricator.haskell.org/D2787

GHC Trac Issues: #8809

Refactoring only

The used-variable calculation for pattern synonyms is a little
tricky, for reasons described in RnBinds
Note [Pattern synonym builders don't yield dependencies]

It was right semantically, but the "unused-variable warning" was
wrong, which led to Trac #12548.

Trac #12425 and #12234 showed up a major and long-standing
bug in the occurrence analyser, whereby it could generate
explonentially large program!

There's a lot of commentary on #12425; and it's all described
in Note [Loop breakers, node scoring, and stability]

I did quite a lot of refactoring to make the code comprehensibe
again (its structure had bit-rotted rather), so the patch
looks bigger than it really is.

Hurrah!

I did a nofib run to check that I hadn't inadertently ruined
anything:

--------------------------------------------------------------------------------
        Program           Size    Allocs   Runtime   Elapsed  TotalMem
--------------------------------------------------------------------------------
          fluid          -0.3%     -1.5%      0.01      0.01     +0.0%
         parser          -0.9%     +0.6%      0.04      0.04     +0.0%
         prolog          -0.1%     +1.2%      0.00      0.00     +0.0%

--------------------------------------------------------------------------------
            Min          -0.9%     -1.5%     -8.6%     -8.7%     +0.0%
            Max          +0.1%     +1.2%     +7.7%     +7.8%     +2.4%
 Geometric Mean          -0.2%     -0.0%     -0.2%     -0.3%     +0.0%

I checked what happened in 'prolog'.  It seems that we have a
recursive data structure something like this

   f :: [blah]
   f x = build (\cn.  ...g...  )

   g :: [blah2]
   g y = ....(foldr k z (f y))....

If we inline 'f' into 'g' we get better fusion than the other
way round, but we don't have any way to spot that at the moment.
(I wonder if we could do worker/wrapper for functions returning
a 'build'?)  It was happening before by a fluke.

Anyway I decided to accept this; it's relatively rare I think.

Previously, it would substitute e for n without an e being around.
I clarify that by naming the scrutinee e.

It's 2.27 (-2), not 1.27.2. Dropped stray '\r'.
Signed-off-by: Sergei Trofimovich <siarheit@google.com>

Maintain in-scope set in deeply_instantiate (Fixes T12549).

lint fixes

Test Plan: validate

Reviewers: simonpj, austin, goldfire, bgamari

Reviewed By: simonpj, bgamari

Subscribers: thomie

Differential Revision: https://phabricator.haskell.org/D2757

GHC Trac Issues: #12549

When rts is forked it doesn't update toplevel handler, so UserInterrupt
exception is sent to Thread1 that doesn't exist in forked process.

We install toplevel handler when fork so signal will be delivered to the
new main thread.

Fixes #12903

Reviewers: simonmar, austin, erikd, bgamari

Reviewed By: bgamari

Subscribers: thomie

Differential Revision: https://phabricator.haskell.org/D2770

GHC Trac Issues: #12903

select() is limited to 1024 file descriptors.  This actually blew up
in a very hard-to-debug way in our production system when using the
hinotify package.

Test Plan:
libraries/tests pass, paricularly hGetBuf001 which exercises this
code.

Reviewers: niteria, erikd, austin, hvr, bgamari

Reviewed By: bgamari

Subscribers: thomie

Differential Revision: https://phabricator.haskell.org/D2785

GHC Trac Issues: #12912

Mention that many GHC testsuite tests will fail with a compiler built with
the quickest profile. See Trac #12141.

The testsuite now requires python >=3.0. See #12909.

textEncodingName is notjust a string, it must be a valid input for
mkTextEncoding, as stated in libraries/base/GHC/IO/Encoding/Types.hs.

Test Plan:
A working latin1 locale is required on the system.  Reason: ghc's
initial locale encoding defaults to ASCII, if either an unknown locale
or unknown charset is used.  For the bug to show up, ghc must start up
using the latin1 encoding.

From main directory in ghc do:
$ ./configure && make clean && make boot && make inplace/bin/ghc-stage2
$ LC_CTYPE="de_DE.ISO-8859-1" ./inplace/bin/ghc-stage2

Before the patch, the last line leads to the exception thrown:
ghc-stage2: mkTextEncoding: does not exist (unknown
encoding:ISO8859-1(checked)//TRANSLIT)

After the patch, ghc-stage2 prints a short usage summary as expected.
Moreover,
$ make test TEST=encoding005
continues to  pass after the patch.

Reviewers: austin, hvr, rwbarton, bgamari

Reviewed By: bgamari

Subscribers: mpickering, rwbarton, thomie

Differential Revision: https://phabricator.haskell.org/D2667

Test Plan: Validate on 64-bit Windows

Reviewers: geekosaur, austin, erikd, Phyx

Reviewed By: Phyx

Subscribers: thomie

Differential Revision: https://phabricator.haskell.org/D2781

The previous attempt updated the comment but not the value. Silly me.

As described in #12865, newer Linux kernels support both MADV_FREE and
MADV_DONTNEED. Previously a runtime would fail to try MADV_DONTNEED if
MADV_FREE failed (e.g. since the kernel which the image is running on
doesn't support the latter). Now we try MADV_DONTNEED if MADV_FREE
failed to ensure that binaries compiled on a kernel supporting MADV_FREE
don't fail on decommit.

Test Plan: Validate

Reviewers: austin, erikd, simonmar

Reviewed By: simonmar

Subscribers: thomie

Differential Revision: https://phabricator.haskell.org/D2780

GHC Trac Issues: #12865

They are not supported by Python 3.0, 3.1, and 3.2 (but are supported by
>= 3.3; silliness!)

Test Plan: Validate on python 3.2

Reviewers: austin

Subscribers: simonpj, thomie

Differential Revision: https://phabricator.haskell.org/D2778

GHC Trac Issues: #12909, #9184

Test Plan: Build `p` way

Reviewers: austin, erikd, simonmar

Reviewed By: simonmar

Subscribers: thomie

Differential Revision: https://phabricator.haskell.org/D2779

Test Plan: Read it, commit it, merge it, ship it

Reviewers: goldfire, bgamari, austin, hvr, simonpj

Reviewed By: simonpj

Subscribers: thomie

Differential Revision: https://phabricator.haskell.org/D2774

GHC Trac Issues: #12907

Reviewers: simonpj, austin, bgamari

Reviewed By: simonpj, bgamari

Subscribers: thomie

Differential Revision: https://phabricator.haskell.org/D2769

GHC Trac Issues: #12901

GHC uses gcc, not ld, for linking. Consequently all flags to be
interpreted by ld need to be prefixed by -optl,-Wl on the GHC command
line.

Test Plan: Validate on OpenBSD

Reviewers: austin, rwbarton

Reviewed By: rwbarton

Subscribers: thomie

Differential Revision: https://phabricator.haskell.org/D2776

Reviewers: austin, bgamari

Reviewed By: bgamari

Subscribers: thomie

Differential Revision: https://phabricator.haskell.org/D2772