GitLab

This patch implements the idea floated in Trac #9858, namely that we
should generate type-representation information at the data type
declaration site, rather than when solving a Typeable constraint.

However, this turned out quite a bit harder than I expected. I still
think it's the right thing to do, and it's done now, but it was quite
a struggle.

See particularly

 * Note [Grand plan for Typeable] in TcTypeable (which is a new module)
 * Note [The overall promotion story] in DataCon (clarifies existing stuff)

The most painful bit was that to generate Typeable instances (ie
TyConRepName bindings) for every TyCon is tricky for types in ghc-prim
etc:

 * We need to have enough data types around to *define* a TyCon
 * Many of these types are wired-in

Also, to minimise the code generated for each data type, I wanted to
generate pure data, not CAFs with unpackCString# stuff floating about.

Performance
~~~~~~~~~~~
Three perf/compiler tests start to allocate quite a bit more. This isn't
surprising, because they all allocate zillions of data types, with
practically no other code, esp. T1969

 * T3294:   GHC allocates 110% more (filed #11030 to track this)
 * T1969:   GHC allocates 30% more
 * T4801:   GHC allocates 14% more
 * T5321FD: GHC allocates 13% more
 * T783:    GHC allocates 12% more
 * T9675:   GHC allocates 12% more
 * T5642:   GHC allocates 10% more
 * T9961:   GHC allocates 6% more

 * T9203:   Program allocates 54% less

I'm treating this as acceptable. The payoff comes in Typeable-heavy
code.

Remaining to do
~~~~~~~~~~~~~~~

 * I think that "TyCon" and "Module" are over-generic names to use for
   the runtime type representations used in GHC.Typeable. Better might be
   "TrTyCon" and "TrModule". But I have not yet done this

 * Add more info the the "TyCon" e.g. source location where it was
   defined

 * Use the new "Module" type to help with Trac Trac #10068

 * It would be possible to generate TyConRepName (ie Typeable
   instances) selectively rather than all the time. We'd need to persist
   the information in interface files. Lacking a motivating reason I have
   not done this, but it would not be difficult.

Refactoring
~~~~~~~~~~~
As is so often the case, I ended up refactoring more than I intended.
In particular

 * In TyCon, a type *family* (whether type or data) is repesented by a
   FamilyTyCon
     * a algebraic data type (including data/newtype instances) is
       represented by AlgTyCon This wasn't true before; a data family
       was represented as an AlgTyCon. There are some corresponding
       changes in IfaceSyn.

     * Also get rid of the (unhelpfully named) tyConParent.

 * In TyCon define 'Promoted', isomorphic to Maybe, used when things are
   optionally promoted; and use it elsewhere in GHC.

 * Cleanup handling of knownKeyNames

 * Each TyCon, including promoted TyCons, contains its TyConRepName, if
   it has one. This is, in effect, the name of its Typeable instance.

Requires update of the haddock submodule.

Differential Revision: https://phabricator.haskell.org/D757

What a disaster this merge was.

Summary:
This reverts commit aecf4a5f.

It turns out the Simons are relying on 'mk/are-validating.mk', see
D1307.

The workflow they are using is:
  * run ./validate
  * find a bug in the compiler
  * try to fix the bug, running 'make 1' (or 'make 2') repeatedly. Because
    of 'mk/are-validating.mk', this uses the same build settings as validate.
  * continue ./validate (--no-clean)

I suggested two alternatives:

  A. run 'make 1 Validating=YES' instead of 'make 1'

     Problem: when running `./validate --fast` or `./validate --hpc`
     instead of a normal `./validate`, validate sets ValidateSpeed and
     ValdateHpc in mk/are-validating.mk. You would for example have to run
     'make 1 Validating=YES ValidateSpeed=FAST' instead of 'make 1' to get the
     same build settings as `./validate --fast`, which is entirely too long and
     error prone.

  B. uncomment `#BuildFlavour=validate` in mk/build.mk, and include
     'mk/validate.mk'.

     Problems:
      * any other settings you have in build.mk will also get used.
      * the distinction between 'mk/validate.mk' and 'mk/build.mk' becomes less
        clear.
      * it is easy to forget to include 'mk/validate.mk'.
      * the build system again doesn't have access to the ValidateSpeed and
        ValdateHpc settings set by validate.

Neither of these two options is entirely satisfactory.

Reviewers: austin, bgamari

Differential Revision: https://phabricator.haskell.org/D1383

This patch implements an extension to pattern synonyms which allows user
to specify pattern synonyms using record syntax. Doing so generates
appropriate selectors and update functions.

=== Interaction with Duplicate Record Fields ===

The implementation given here isn't quite as general as it could be with
respect to the recently-introduced `DuplicateRecordFields` extension.

Consider the following module:

    {-# LANGUAGE DuplicateRecordFields #-}
    {-# LANGUAGE PatternSynonyms #-}

    module Main where

    pattern S{a, b} = (a, b)
    pattern T{a}    = Just a

    main = do
      print S{ a = "fst", b = "snd" }
      print T{ a = "a" }

In principle, this ought to work, because there is no ambiguity. But at
the moment it leads to a "multiple declarations of a" error. The problem
is that pattern synonym record selectors don't do the same name mangling
as normal datatypes when DuplicateRecordFields is enabled. They could,
but this would require some work to track the field label and selector
name separately.

In particular, we currently represent datatype selectors in the third
component of AvailTC, but pattern synonym selectors are just represented
as Avails (because they don't have a corresponding type constructor).
Moreover, the GlobalRdrElt for a selector currently requires it to have
a parent tycon.

(example due to Adam Gundry)

=== Updating Explicitly Bidirectional Pattern Synonyms ===

Consider the following

```
pattern Silly{a} <- [a] where
  Silly a = [a, a]

f1 = a [5] -- 5

f2 = [5] {a = 6} -- currently [6,6]
```

=== Fixing Polymorphic Updates ===

They were fixed by adding these two lines in `dsExpr`. This might break
record updates but will be easy to fix.

```
+ ; let req_wrap = mkWpTyApps (mkTyVarTys univ_tvs)

- , pat_wrap = idHsWrapper }
+, pat_wrap = req_wrap }
```

=== Mixed selectors error ===

Note [Mixed Record Field Updates]
~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Consider the following pattern synonym.

    data MyRec = MyRec { foo :: Int, qux :: String }

    pattern HisRec{f1, f2} = MyRec{foo = f1, qux=f2}

This allows updates such as the following

    updater :: MyRec -> MyRec
    updater a = a {f1 = 1 }

It would also make sense to allow the following update (which we
reject).

    updater a = a {f1 = 1, qux = "two" } ==? MyRec 1 "two"

This leads to confusing behaviour when the selectors in fact refer the
same field.

    updater a = a {f1 = 1, foo = 2} ==? ???

For this reason, we reject a mixture of pattern synonym and normal
record selectors in the same update block. Although of course we still
allow the following.

    updater a = (a {f1 = 1}) {foo = 2}

    > updater (MyRec 0 "str")
    MyRec 2 "str"

This reverts commit 9fc2d777.

This appears to cause interface file issues during rebuilds. Punting
back to @thomie for further investigation.

Users are sometimes confused why their test doesn't run. It is usually
because of a misspelled testname, for example using 'TEST=1234' instead
of 'TEST=T1234'. After this patch it is hopefully more clear what the
problem is, showing:

    ERROR: tests not found: ['1234']

Instead of:

    0 total tests, which gave rise to
    0 test cases, of which
    0 were skipped

Reviewed by: austin, bgamari

Differential Revision: https://phabricator.haskell.org/D1388

Signed-off-by: Edward Z. Yang <ezyang@cs.stanford.edu>

Sed on OS X does not understand 's/[0-9]\+ bytes/NUM bytes/g' but
sed on Linux and OS X do understand 's/[0-9]* bytes/NUM bytes/g'.

Test Plan: Run all rts/T9579 tests on Linux and Mac

Reviewers: thomie, austin, bgamari

Differential Revision: https://phabricator.haskell.org/D1394

Drop support for old OS X (OS X 10.2 and earlier) dynamic linking.
10.3 was released in 2003.

Test Plan: Validate on OS X and Linux.

Reviewers: bgamari, thomie, austin

Reviewed By: thomie, austin

Differential Revision: https://phabricator.haskell.org/D1393

This is done now to prepare for #11026

This is needed to prepare for #11026 as these updates
relax the upper bounds on `base` to allow for `base-4.9.0.0`

This update contains no code-changes to terminfo/haskeline yet

This patch swaps the order of provided and required constraints in
a pattern signature, so it now goes

      pattern P :: req => prov => t1 -> ... tn -> res_ty

See the long discussion in Trac #10928.

I think I have found all the places, but I could have missed something
particularly in comments.

There is a Haddock changes; so a submodule update.

Summary:
The current scheme in rules/distdir-way-opts.mk is something like this:
  GHC_LD_OPTS      = MOST_HC_OPTS + ALL_LD_OPTS
  MOST_DIR_HC_OPTS = MOST_HC_OPTS + -odir,-hidir,-stubdir
  ALL_HC_OPTS      = MOST_DIR_HC_OPTS +
                        -hisuf,-osuf,-hcsuf,-split-objs,-dynamic-too

Notice that both ALL_HC_OPTS and GHC_LD_OPTS include MOST_HC_OPTS, and
currently both got added when linking. Adding MOST_HC_OPTS twice results
in overly long and hard to decipher command lines (and build logs). This
commit fixes that.

Afaik, -odir,-hidir,-stubdir,-hisuf,-osuf,-hcsuf,-spit-objs,-dynamic-too
are all not needed when linking, so this change should be safe to make.
GHC_LD_OPTS is for linking, ALL_HC_OPTS is for compiling.

ALL_HC_OPTS was added to the linking commands in
37a6a52f, to make sure
-no-user-package-conf would be in the options list. It still is after
this change.

Reviewers: austin, bgamari

Differential Revision: https://phabricator.haskell.org/D1379

Previously unboundKey and fromIntegerClassOpKey were sharing a Unique
Reassign unboundKey to `mkPreludeMiscIdUnique 158`

Signed-off-by: Edward Z. Yang <ezyang@cs.stanford.edu>

LEADING_UNDERSCORE is defined in a header file, so we need to #include
that file where we use LEADING_UNDERSCORE.

Reviewed By: austin

Differential Revision: https://phabricator.haskell.org/D1382

Add -fworker-wrapper flag which enables the worker-wrapper transformation. It
is implied by -O.

The expected users of this flag, which includes myself, are GHC API users.  In
my Haskell-to-Hardware compiler, which uses the GHC API, I have seen no
benifits of the worker-wrapper transformation. It does however induce longer
compilation times.

Further discussion can be seen here:
https://mail.haskell.org/pipermail/ghc-devs/2015-October/010096.html

Reviewed By: austin

Differential Revision: https://phabricator.haskell.org/D1372

GHC Trac Issues: #11020

`fsEnvElts :: FastStringEnv a -> [a]` returns a list of `[a]` in the order of
`Unique`s which is arbitrary. In this case it gives a list of record fields in
arbitrary order, from which we then extract the field labels to contribute to
the record fingerprint. The arbitrary ordering of field labels introduces
unnecessary nondeterminism in interface files as demonstrated by the test case.

We sort `FastString` here. It's safe, because the only way that the `Unique`
associated with the `FastString` is used in comparison is for equality. If the
`Unique`s are different it fallbacks to comparing the actual `ByteString`.

Reviewed By: ezyang, thomie, bgamari, austin

Differential Revision: https://phabricator.haskell.org/D1373

GHC Trac Issues: #4012

This fixes commit d1d8704c,
which somehow was mangled during merge.

To get reproducible/deterministic builds, the way that the Uniques are
assigned shouldn't matter. This allows to test for that.

It add 2 new flags:

* `-dinitial-unique`
* `-dunique-increment`

And by varying these you can get interesting effects:

* `-dinitial-unique=0 -dunique-increment 1` - current sequential
  UniqSupply

* `-dinitial-unique=16777215 -dunique-increment -1` - UniqSupply that
  generates in decreasing order

* `-dinitial-unique=1 -dunique-increment PRIME` - where PRIME big enough
  to overflow often - nonsequential order

I haven't proven the usefullness of the last one yet and it's the reason
why we have to mask the bits with `0xFFFFFF` in `genSym`, so I can
remove it if it becomes contentious.

Test Plan: validate on harbormaster

Reviewers: simonmar, austin, ezyang, bgamari

Reviewed By: austin, bgamari

Subscribers: thomie

Differential Revision: https://phabricator.haskell.org/D1360

GHC Trac Issues: #4012

Reviewers: austin, thomie, bgamari

Reviewed By: bgamari

Differential Revision: https://phabricator.haskell.org/D1376

GHC Trac Issues: #10848

This silences warnings from Sphinx of the form,

    docs/users_guide/ghc.rst:: WARNING: document isn't included in any toctree

Fixes Trac #10997

Merge to stable branch

Test Plan: validate

Reviewers: austin, thomie, bgamari

Reviewed By: thomie

Subscribers: thomie

Differential Revision: https://phabricator.haskell.org/D1349

GHC Trac Issues: #10970

Reviewers: austin, bgamari

Reviewed By: bgamari

Subscribers: thomie

Differential Revision: https://phabricator.haskell.org/D1367

GHC Trac Issues: #10848

Build system support for Cygwin was removed in b6be81b8.

Test Plan:
 - Validate on x86_64/linux
 - Cross-compile rts/RtsSymbols.c and rts/Linker.c to Windows using the
   i686-w64-mingw32-gcc and x86_64-w64-mingw32-gcc cross compilers.

Reviewers: hvr, austin, bgamari

Reviewed By: bgamari

Subscribers: thomie

Differential Revision: https://phabricator.haskell.org/D1371

Test Plan: harbormaster

Reviewers: austin, bgamari

Reviewed By: bgamari

Subscribers: thomie

Differential Revision: https://phabricator.haskell.org/D1374

$ git grep -n "typedef struct LibDwSession_ "
rts/Libdw.c:63:typedef struct LibDwSession_ LibDwSession;
rts/Libdw.h:22:typedef struct LibDwSession_ LibDwSession;

This trips up (e.g.) GCC v4.4.7.

Test Plan: Validate.

Reviewers: austin

Reviewed By: austin

Subscribers: thomie

Differential Revision: https://phabricator.haskell.org/D1352

GHC Trac Issues: #10370

It is difficult for GHC developers to know if they have broken the API
Annotations.

This patch provides a utility that can be used as a test to show up
errors in the API Annotations.

It is based on the current tests for ghc-api/annotations which can parse
a file using the just-built GHC API, and check that no annotations are
disconnected from the ParsedSource in the output.

In addition, it should be able to dump the annotations to a file, so a
new feature developer can check that all changes to the parser do
provide annotations.

Trac ticket: #10917

Test Plan: ./validate

Reviewers: hvr, thomie, austin, bgamari

Reviewed By: bgamari

Differential Revision: https://phabricator.haskell.org/D1368

GHC Trac Issues: #10917

Test Plan:
 - Build whole of GHC on Linux.
 - Cross-compile rts/RtsSymbols.c and rts/Linker.c to Windows using the
   i686-w64-mingw32-gcc and x86_64-w64-mingw32-gcc cross compilers.

Reviewers: bgamari, awson, austin

Subscribers: thomie

Differential Revision: https://phabricator.haskell.org/D1365