GitLab

The splitter is an evil Perl script that processes assembler code.
Its job can be done better by the linker's --gc-sections flag. GHC
passes this flag to the linker whenever -split-sections is passed on
the command line.

This is based on @DemiMarie's D2768.

Fixes Trac #11315
Fixes Trac #9832
Fixes Trac #8964
Fixes Trac #8685
Fixes Trac #8629

This addresses Trac #16208 by marking newtype wrapper
unfoldings as compulsory.

Furthermore, we can remove the special case for newtypes
in exprIsConApp_maybe (introduced in 7833cf40).

Test Runner: don't show missing baseline warning for performance tests with expected changes on the current commit.

Trac #16359

This patch fixes two rather gnarly test cases:
  * Trac #16342 (mutual recursion)
    See Note [Tricky scoping in generaliseTcTyCon]

  * Trac #16221 (shadowing)
    See Note [Unification variables need fresh Names]

The main changes are:

* Substantial reworking of TcTyClsDecls.generaliseTcTyCon
  This is the big change, and involves the rather tricky
  function TcHsSyn.zonkRecTyVarBndrs.

  See Note [Inferring kinds for type declarations] and
  Note [Tricky scoping in generaliseTcTyCon] for the details.

* bindExplicitTKBndrs_Tv and bindImplicitTKBndrs_Tv both now
  allocate /freshly-named/ unification variables. Indeed, more
  generally, unification variables are always fresh; see
  Note [Unification variables need fresh Names] in TcMType

* Clarify the role of tcTyConScopedTyVars.
  See Note [Scoped tyvars in a TcTyCon] in TyCon

As usual, this dragged in some more refactoring:

* Renamed TcMType.zonkTyCoVarBndr to zonkAndSkolemise

* I renamed checkValidTelescope to checkTyConTelescope;
  it's only used on TyCons, and indeed takes a TyCon as argument.

* I folded the slightly-mysterious reportFloatingKvs into
  checkTyConTelescope. (Previously all its calls immediately
  followed a call to checkTyConTelescope.)  It makes much more
  sense there.

* I inlined some called-once functions to simplify
  checkValidTyFamEqn. It's less spaghetti-like now.

* This patch also fixes Trac #16251.  I'm not quite sure why #16251
  went wrong in the first place, nor how this patch fixes it, but
  hey, it's good, and life is short.

This reverts commit 0e2d300a.

This reverts commit e8a08f40.

RyanGlScott observed in https://github.com/haskell/haddock/issues/1030
that running Haddock tests in GHC's testsuite left some `.hi` files
around in `utils/haddock`. This should fix that problem.

Otherwise these tests break spuriously when core libraries are compiled
with source notes.

These have been moved to the ghc/ci-images project.

Python's split() function is used to split on all white space.

The initial motivation for this is to have a chance to run the binary
distribution rules in our Windows CI without having to install
sphinx-build and xelatex there, while retaining the ability to
generate haddocks. I just ended up extending this idea a little bit so
as to have control over whether we build haddocks, (sphinx) HTML manuals,
(sphinx) PDF manuals and (sphinx) manpages.

The following previously fatal lexer errors are now non-fatal:

  * errors about enabling `LambdaCase`
  * errors about enabling `NumericUnderscores`
  * errors about having valid characters in primitive strings

See #16270

This implements GHC proposal 35
(https://github.com/ghc-proposals/ghc-proposals/blob/master/proposals/0035-forall-arrow.rst)
by adding the ability to write kinds with
visible dependent quantification (VDQ).

Most of the work for supporting VDQ was actually done _before_ this
patch. That is, GHC has been able to reason about kinds with VDQ for
some time, but it lacked the ability to let programmers directly
write these kinds in the source syntax. This patch is primarly about
exposing this ability, by:

* Changing `HsForAllTy` to add an additional field of type
  `ForallVisFlag` to distinguish between invisible `forall`s (i.e,
  with dots) and visible `forall`s (i.e., with arrows)
* Changing `Parser.y` accordingly

The rest of the patch mostly concerns adding validity checking to
ensure that VDQ is never used in the type of a term (as permitting
this would require full-spectrum dependent types). This is
accomplished by:

* Adding a `vdqAllowed` predicate to `TcValidity`.
* Introducing `splitLHsSigmaTyInvis`, a variant of `splitLHsSigmaTy`
  that only splits invisible `forall`s. This function is used in
  certain places (e.g., in instance declarations) to ensure that GHC
  doesn't try to split visible `forall`s (e.g., if it tried splitting
  `instance forall a -> Show (Blah a)`, then GHC would mistakenly
  allow that declaration!)

This also updates Template Haskell by introducing a new `ForallVisT`
constructor to `Type`.

Fixes #16326. Also fixes #15658 by documenting this feature in the
users' guide.

This adds trace messages that include the processes name and as such
make debugging and following the communication easier.

It also adds a note regarding the fwd*Call proxy-communication logic
between the proxy and the slave.

The proxy will now also poll for 60s to wait for the remote iserv
to come up. (Alternatively you can start the remote process
beforehand; and just have iserv-proxy connect to it)

hie002 is a performance test that used to fail unpredictably:

	max_bytes_used Decrease from x86_64-linux-deb9-debug baseline @ HEAD~2:
	    Expected    hie002 (normal) max_bytes_used: 1190923992.0 +/-20%
	    Lower bound hie002 (normal) max_bytes_used:    952739193
	    Upper bound hie002 (normal) max_bytes_used:   1429108791
	    Actual      hie002 (normal) max_bytes_used:    726270784
	    Deviation   hie002 (normal) max_bytes_used:        -39.0 %
	peak_megabytes_allocated Decrease from x86_64-linux-deb9-debug baseline @ HEAD~2:
	    Expected    hie002 (normal) peak_megabytes_allocated: 2538.0 +/-20%
	    Lower bound hie002 (normal) peak_megabytes_allocated:   2030
	    Upper bound hie002 (normal) peak_megabytes_allocated:   3046
	    Actual      hie002 (normal) peak_megabytes_allocated:   1587
	    Deviation   hie002 (normal) peak_megabytes_allocated:  -37.5 %
	*** unexpected stat test failure for hie002(normal)

'max_bytes_used' and 'peak_megabytes_allocated' are too unstable without careful
control of the runtime configuration. We fix this by using a more predictable
metric, 'bytes allocated'.

In the work stealing queue a load-load-barrier is required to ensure
that a read of queue data cannot be reordered before a read of the
bottom pointer into the queue.

The added load-load-barrier ensures that the ordering of writes enforced
at the end of `pushWSDeque` is also respected in the order of reads in
`stealWSDeque_`. In other words, when reading `q->bottom` we want to make
sure that we see the updates to `q->elements`.

Fixes #13633

Implements GHC Proposal #24: .../ghc-proposals/blob/master/proposals/0024-no-kind-vars.rst
Fixes Trac #16334, Trac #16315

With this patch, scoping rules for type and kind variables have been
unified: kind variables no longer receieve special treatment. This
simplifies both the language and the implementation.

User-facing changes
-------------------

* Kind variables are no longer implicitly quantified when an explicit
  forall is used:

    p ::             Proxy (a :: k)    -- still accepted
    p :: forall k a. Proxy (a :: k)    -- still accepted
    p :: forall   a. Proxy (a :: k)    -- no longer accepted

  In other words, now we adhere to the "forall-or-nothing" rule more
  strictly.

  Related function: RnTypes.rnImplicitBndrs

* The -Wimplicit-kind-vars warning has been deprecated.

* Kind variables are no longer implicitly quantified in constructor
  declarations:

    data T a        = T1 (S (a :: k) | forall (b::k). T2 (S b)  -- no longer accepted
    data T (a :: k) = T1 (S (a :: k) | forall (b::k). T2 (S b)  -- still accepted

  Related function: RnTypes.extractRdrKindSigVars

* Implicitly quantified kind variables are no longer put in front of
  other variables:

    f :: Proxy (a :: k) -> Proxy (b :: j)

    f :: forall k j (a :: k) (b :: j). Proxy a -> Proxy b   -- old order
    f :: forall k (a :: k) j (b :: j). Proxy a -> Proxy b   -- new order

  This is a breaking change for users of TypeApplications. Note that
  we still respect the dpendency order: 'k' before 'a', 'j' before 'b'.
  See "Ordering of specified variables" in the User's Guide.

  Related function: RnTypes.rnImplicitBndrs

* In type synonyms and type family equations, free variables on the RHS
  are no longer implicitly quantified unless used in an outermost kind
  annotation:

    type T = Just (Nothing :: Maybe a)         -- no longer accepted
    type T = Just Nothing :: Maybe (Maybe a)   -- still accepted

  The latter form is a workaround due to temporary lack of an explicit
  quantification method. Ideally, we would write something along these
  lines:

    type T @a = Just (Nothing :: Maybe a)

  Related function: RnTypes.extractHsTyRdrTyVarsKindVars

* Named wildcards in kinds are fixed (Trac #16334):

    x :: (Int :: _t)    -- this compiles, infers (_t ~ Type)

  Related function: RnTypes.partition_nwcs

Implementation notes
--------------------

* One of the key changes is the removal of FKTV in RnTypes:

  - data FreeKiTyVars = FKTV { fktv_kis    :: [Located RdrName]
  -                          , fktv_tys    :: [Located RdrName] }
  + type FreeKiTyVars = [Located RdrName]

  We used to keep track of type and kind variables separately, but
  now that they are on equal footing when it comes to scoping, we
  can put them in the same list.

* extract_lty and family are no longer parametrized by TypeOrKind,
  as we now do not distinguish kind variables from type variables.

* PatSynExPE and the related Note [Pattern synonym existentials do not scope]
  have been removed (Trac #16315). With no implicit kind quantification,
  we can no longer trigger the error.

* reportFloatingKvs and the related Note [Free-floating kind vars]
  have been removed. With no implicit kind quantification,
  we can no longer trigger the error.

14586f5d removed this by accident.

We use "touch -r" to set modification timestamps, which leads to precision loss
on Darwin. For example,

   before: 2019-02-25 01:11:23.807627350 +0300
   after:  2019-02-25 01:11:23.807627000 +0300
                                     ^^^
This means we can't trick GHCi into thinking the file hasn't been changed by
restoring its old timestamp, as we cannot faithfully restore all digits.

The solution is to nullify the insignificant digits before the first :load

See Trac #15072, Trac #16349, Trac #16350

Fixes #14037.

Metric Decrease:
    T9872b
    T9872d

Reviewers: bgamari, simonpj, hvr

Reviewed By: simonpj

Subscribers: AndreasK, simonpj, osa1, dfeuer, rwbarton, carter

GHC Trac Issues: #14037

Differential Revision: https://phabricator.haskell.org/D5249

We no longer put class variables in front,
see "Taming the Kind Inference Monster"

(also fix some markup issues)

While playing around with late lambda lifting, I realised that
StgLamLift.Analysis doesn't consider the removed closure header in its
allocation estimations.

That's because contrary to what I thought, the total word count returned
by `mkVirtHeapOffsets` doesn't include the size of the closure header.

We just add the header size manually now.

These are largely redundant as they are covered by x86_64-deb9.

This was causing gitlab to not report from builds as failing. It also
highlighted a problem with the LLVM tests where some of the external
interpreter tests are failing.

`--supported-languages` must only advertise language extensions
which are supported by the compiler in order for tooling such
as Cabal relying on this signalling not to behave incorrectly.

Fixes #16331

This patch removes 'HsArrApp' and 'HsArrForm' from 'HsExpr' by
introducing a new ambiguity resolution system in the parser.

Problem: there are places in the grammar where we do not know whether we
are parsing an expression or a command:

	proc x -> do { (stuff) -< x }   -- 'stuff' is an expression
	proc x -> do { (stuff) }        -- 'stuff' is a command

Until we encounter arrow syntax (-<) we don't know whether to parse
'stuff' as an expression or a command.

The old solution was to parse as HsExpr always, and rejig later:

	checkCommand :: LHsExpr GhcPs -> P (LHsCmd GhcPs)

This meant polluting 'HsExpr' with command-related constructors. In
other words, limitations of the parser were affecting the AST, and
all other code (the renamer, the typechecker) had to deal with these
extra constructors by panicking.

We fix this abstraction leak by parsing into an intermediate
representation, 'ExpCmd':

	data ExpCmdG b where
	  ExpG :: ExpCmdG HsExpr
	  CmdG :: ExpCmdG HsCmd

	type ExpCmd = forall b. ExpCmdG b -> PV (Located (b GhcPs))

	checkExp :: ExpCmd -> PV (LHsExpr GhcPs)
	checkCmd :: ExpCmd -> PV (LHsCmd GhcPs)
	checkExp f = f ExpG  -- interpret as an expression
	checkCmd f = f CmdG  -- interpret as a command

See Note [Ambiguous syntactic categories] for details.

Now the intricacies of parsing have no effect on the hsSyn AST when it
comes to the expression/command ambiguity.

Future work: apply the same principles to the expression/pattern
ambiguity.

The big payload of this patch is:

  Add an AnonArgFlag to the FunTy constructor
  of Type, so that
    (FunTy VisArg   t1 t2) means (t1 -> t2)
    (FunTy InvisArg t1 t2) means (t1 => t2)

The big payoff is that we have a simple, local test to make
when decomposing a type, leading to many fewer calls to
isPredTy. To me the code seems a lot tidier, and probably
more efficient (isPredTy has to take the kind of the type).

See Note [Function types] in TyCoRep.

There are lots of consequences

* I made FunTy into a record, so that it'll be easier
  when we add a linearity field, something that is coming
  down the road.

* Lots of code gets touched in a routine way, simply because it
  pattern matches on FunTy.

* I wanted to make a pattern synonym for (FunTy2 arg res), which
  picks out just the argument and result type from the record. But
  alas the pattern-match overlap checker has a heart attack, and
  either reports false positives, or takes too long.  In the end
  I gave up on pattern synonyms.

  There's some commented-out code in TyCoRep that shows what I
  wanted to do.

* Much more clarity about predicate types, constraint types
  and (in particular) equality constraints in kinds.  See TyCoRep
  Note [Types for coercions, predicates, and evidence]
  and Note [Constraints in kinds].

  This made me realise that we need an AnonArgFlag on
  AnonTCB in a TyConBinder, something that was really plain
  wrong before. See TyCon Note [AnonTCB InivsArg]

* When building function types we must know whether we
  need VisArg (mkVisFunTy) or InvisArg (mkInvisFunTy).
  This turned out to be pretty easy in practice.

* Pretty-printing of types, esp in IfaceType, gets
  tidier, because we were already recording the (->)
  vs (=>) distinction in an ad-hoc way.  Death to
  IfaceFunTy.

* mkLamType needs to keep track of whether it is building
  (t1 -> t2) or (t1 => t2).  See Type
  Note [mkLamType: dictionary arguments]

Other minor stuff

* Some tidy-up in validity checking involving constraints;
  Trac #16263

Remove a bogus assertion in FamInst.newFamInst
(There is a comment to explain.)

This fixes Trac #16112.