GitLab

These tweaks affect the inner loop of simplifyArgsWorker, which
in turn is called from the flattener in Flatten.hs.  This is
a key perf bottleneck to T9872{a,b,c,d}.

These two small changes have a modest but useful benefit.
No change in functionality whatsoever.

Relates to #18354

This bug, revealed by #18347, is just a missing update to
sc_hole_ty in simplCast.  I'd missed a code path when I
made the recentchanges in

    commit 6d49d5be
    Author: Simon Peyton Jones <simonpj@microsoft.com>
    Date:   Thu May 21 12:53:35 2020 +0100

    Implement cast worker/wrapper properly

The fix is very easy.

Two other minor changes

* Tidy up in SimpleOpt.simple_opt_expr. In fact I think this is an
  outright bug, introduced in the fix to #18112: we were simplifying
  the same coercion twice *with the same substitution*, which is just
  wrong.  It'd be a hard bug to trigger, so I just fixed it; less code
  too.

* Better debug printing of ApplyToVal

Zonk residual constraints in checkForExistence to reveal user type
errors.

Previously when `:instances` was used with instances that have TypeError
constraints the result would look something like:

instance [safe] s0 => Err 'A -- Defined at ../Bug2.hs:8:10

whereas after zonking, `:instances` now sees the `TypeError` and
properly eliminates the constraint from the results.

Metric Decrease:
    T12150
    T12234

Previously, if a .cmm file *not in the RTS* contained something like:

```cmm
section "rodata" { msg : bits8[] "Test\n"; }
```

It would get compiled by CmmToC into:

```c
ERW_(msg);
const char msg[] = "Test\012";
```

and fail with:

```
/tmp/ghc32129_0/ghc_4.hc:5:12: error:
     error: conflicting types for \u2018msg\u2019
     const char msg[] = "Test\012";
                ^~~

In file included from /tmp/ghc32129_0/ghc_4.hc:3:0: error:

/tmp/ghc32129_0/ghc_4.hc:4:6: error:
     note: previous declaration of \u2018msg\u2019 was here
     ERW_(msg);
          ^

/builds/hsyl20/ghc/_build/install/lib/ghc-8.11.0.20200605/lib/../lib/x86_64-linux-ghc-8.11.0.20200605/rts-1.0/include/Stg.h:253:46: error:
     note: in definition of macro \u2018ERW_\u2019
     #define ERW_(X)   extern       StgWordArray (X)
                                                  ^
```

See the rationale for this on https://gitlab.haskell.org/ghc/ghc/-/wikis/commentary/compiler/backends/ppr-c#prototypes

Now we don't generate these extern declarations (ERW_, etc.) for
top-level data. It shouldn't change anything for the RTS (the only place
we use .cmm files) as it is already special cased in
`GHC.Cmm.CLabel.needsCDecl`. And hand-written Cmm can use explicit
extern declarations when needed.

Note that it allows `cgrun069` test to pass with CmmToC (cf #15467).

We don't want to save both Fn and Dn register sets on x86-64 as they are
aliased to the same arch register (XMMn).

Moreover, when SAVE_STGREGS was used in conjunction with `jump foo [*]`
which makes a set of Cmm registers alive so that they cover all arch
registers used to pass parameter, we could have Fn, Dn and XMMn alive at
the same time. It made the LLVM code generator choke (see #17920).

Now `SAVE_REGS/RESTORE_REGS` and `jump foo [*]` use the same set of
registers.

We look up modules by their name, and not their contents. There is no
way to separately reference a signature vs regular module; you get what
you get. Only boot files can be referenced indepenently with `import {-#
SOURCE #-}`.

tablesNextToCode is a platform setting and doesn't belong into DynFlags
(#17957). Doing this is also a prerequisite to fix #14335 where we deal
with two platforms (target and host) that may have different platform
settings.

It avoids having to use DynFlags to reach for pprUserLength.

Valgrind report of the bug when running the test `linker_unload`:

    ==29666== Conditional jump or move depends on uninitialised value(s)
    ==29666==    at 0x369C5B4: setOcInitialStatus (Linker.c:1305)
    ==29666==    by 0x369C6C5: mkOc (Linker.c:1347)
    ==29666==    by 0x36C027A: loadArchive_ (LoadArchive.c:522)
    ==29666==    by 0x36C0600: loadArchive (LoadArchive.c:626)
    ==29666==    by 0x2C144CD: ??? (in /home/omer/haskell/ghc_2/testsuite/tests/rts/linker/linker_unload.run/linker_unload)
    ==29666==
    ==29666== Conditional jump or move depends on uninitialised value(s)
    ==29666==    at 0x369C5B4: setOcInitialStatus (Linker.c:1305)
    ==29666==    by 0x369C6C5: mkOc (Linker.c:1347)
    ==29666==    by 0x369C9F6: preloadObjectFile (Linker.c:1507)
    ==29666==    by 0x369CA8D: loadObj_ (Linker.c:1536)
    ==29666==    by 0x369CB17: loadObj (Linker.c:1557)
    ==29666==    by 0x3866BC: main (linker_unload.c:33)

The problem is `mkOc` allocates a new `ObjectCode` and calls
`setOcInitialStatus` without initializing the `status` field.
`setOcInitialStatus` reads the field as first thing:

    static void setOcInitialStatus(ObjectCode* oc) {
        if (oc->status == OBJECT_DONT_RESOLVE)
          return;

        if (oc->archiveMemberName == NULL) {
            oc->status = OBJECT_NEEDED;
        } else {
            oc->status = OBJECT_LOADED;
        }
    }

`setOcInitialStatus` is unsed in two places for two different purposes:
in `mkOc` where we don't have the `status` field initialized yet (`mkOc`
is supposed to initialize it), and `loadOc` where we do have `status`
field initialized and we want to update it. Instead of splitting the
function into two functions which are both called just once I inline the
functions in the use sites and remove it.

Fixes #18342

This test performs little work, so the most minor allocation
changes often cause the test to fail.

Increasing the threshold to 2% should help with this.

[skip ci]

* support ghc-bignum backend selection in flavours and command-line

* support ghc-bignum "--check" flag (compare results of selected backend
  against results of the native one) in flavours and command-line (e.g.
  pass --bignum=check-gmp" to check the "gmp" backend)

* remove the hack to workaround #15286

* build GMP only when the gmp backend is used

* remove hacks to workaround `text` package flags about integer-*. We
  fix `text` to use ghc-bignum unconditionally in another patch

* support detection of slow ghc-bignum backend (to replace the detection
  of integer-simple use). There are still some test cases that the
  native backend doesn't handle efficiently enough.

* remove tests for GMP only functions that have been removed from
  ghc-bignum

* fix test results showing dependent packages (e.g. integer-gmp) or
  showing suggested instances

* fix test using Integer/Natural API or showing internal names

* replace integer-* package selection with ghc-bignum backend selection

* GHC.Natural isn't implemented in `base` anymore. It is provided by
  ghc-bignum in GHC.Num.Natural. It means that we can safely use Natural
  primitives in `base` without fearing issues with built-in rewrite
  rules (cf #15286)

* `base` doesn't conditionally depend on an integer-* package anymore,
  it depends on ghc-bignum

* Some duplicated code in integer-* can now be factored in GHC.Float

* ghc-bignum tries to use a uniform naming convention so most of the
  other changes are renaming

Thanks to ghc-bignum, the compiler can be simplified:

* Types and constructors of Integer and Natural can be wired-in. It
  means that we don't have to query them from interfaces. It also means
  that numeric literals don't have to carry their type with them.

* The same code is used whatever ghc-bignum backend is enabled. In
  particular, conversion of bignum literals into final Core expressions
  is now much more straightforward. Bignum closure inspection too.

* GHC itself doesn't depend on any integer-* package anymore

* The `integerLibrary` setting is gone.

ghc-bignum is a newer package that aims to replace the legacy
integer-simple and integer-gmp packages.

* it supports several backends. In particular GMP is still supported and
  most of the code from integer-gmp has been merged in the "gmp"
  backend.

* the pure Haskell "native" backend is new and is much faster than the
  previous pure Haskell implementation provided by integer-simple

* new backends are easier to write because they only have to provide a
  few well defined functions. All the other code is common to all
  backends. In particular they all share the efficient small/big number
  distinction previously used only in integer-gmp.

* backends can all be tested against the "native" backend with a simple
  Cabal flag. Backends are only allowed to differ in performance, their
  results should be the same.

* Add `integer-gmp` compat package: provide some pattern synonyms and
  function aliases for those in `ghc-bignum`. It is intended to avoid
  breaking packages that depend on `integer-gmp` internals.

Update submodules: text, bytestring

Metric Decrease:
    Conversions
    ManyAlternatives
    ManyConstructors
    Naperian
    T10359
    T10547
    T10678
    T12150
    T12227
    T12234
    T12425
    T13035
    T13719
    T14936
    T1969
    T4801
    T4830
    T5237
    T5549
    T5837
    T8766
    T9020
    parsing001
    space_leak_001
    T16190
    haddock.base

On ARM and i386, T17499 regresses (+6% > 5%).
On x86_64 unregistered, T13701 sometimes regresses (+2.2% > 2%).

Metric Increase:
    T17499
    T13701

integer-simple uses lists of words (`[Word]`) to represent big numbers
instead of ByteArray#:

   * it is less efficient than the newer ghc-bignum native backend

   * it isn't compatible with the big number representation that is now
     shared by all the ghc-bignum backends (based on the one that was
     used only in integer-gmp before).

As a consequence, we simply drop integer-simple

This implements several general performance improvements to GHC,
to offset the effect of the linear types change.

General optimisations:
- Add a `coreFullView` function which iterates `coreView` on the
  head. This avoids making function recursive solely because the
  iterate `coreView` themselves. As a consequence, this functions can
  be inlined, and trigger case-of-known constructor (_e.g._
  `kindRep_maybe`, `isLiftedRuntimeRep`, `isMultiplicityTy`,
  `getTyVar_maybe`, `splitAppTy_maybe`, `splitFunType_maybe`,
  `tyConAppTyCon_maybe`). The common pattern about all these functions
  is that they are almost always used as views, and immediately
  consumed by a case expression. This commit also mark them asx `INLINE`.
- In `subst_ty` add a special case for nullary `TyConApp`, which avoid
  allocations altogether.
- Use `mkTyConApp` in `subst_ty` for the general `TyConApp`. This
  required quite a bit of module shuffling.
  case. `myTyConApp` enforces crucial sharing, which was lost during
  substitution. See also !2952 .
- Make `subst_ty` stricter.
- In `eqType` (specifically, in `nonDetCmpType`), add a special case,
  tested first, for the very common case of nullary `TyConApp`.
  `nonDetCmpType` has been made `INLINE` otherwise it is actually a
  regression. This is similar to the optimisations in !2952.

Linear-type specific optimisations:
- Use `tyConAppTyCon_maybe` instead of the more complex `eqType` in
  the definition of the pattern synonyms `One` and `Many`.
- Break the `hs-boot` cycles between `Multiplicity.hs` and `Type.hs`:
  `Multiplicity` now import `Type` normally, rather than from the
  `hs-boot`. This way `tyConAppTyCon_maybe` can inline properly in the
  `One` and `Many` pattern synonyms.
- Make `updateIdTypeAndMult` strict in its type and multiplicity
- The `scaleIdBy` gets a specialised definition rather than being an
  alias to `scaleVarBy`
- `splitFunTy_maybe` is given the type `Type -> Maybe (Mult, Type,
  Type)` instead of `Type -> Maybe (Scaled Type, Type)`
- Remove the `MultMul` pattern synonym in favour of a view `isMultMul`
  because pattern synonyms appear not to inline well.
- in `eqType`, in a `FunTy`, compare multiplicities last: they are
  almost always both `Many`, so it helps failing faster.
- Cache `manyDataConTy` in `mkTyConApp`, to make sure that all the
  instances of `TyConApp ManyDataConTy []` are physically the same.

This commit has been authored by
* Richard Eisenberg
* Krzysztof Gogolewski
* Arnaud Spiwack

Metric Decrease:
    haddock.base
    T12227
    T12545
    T12990
    T1969
    T3064
    T5030
    T9872b

Metric Increase:
    haddock.base
    haddock.Cabal
    haddock.compiler
    T12150
    T12234
    T12425
    T12707
    T13035
    T13056
    T15164
    T16190
    T18304
    T1969
    T3064
    T3294
    T5631
    T5642
    T5837
    T6048
    T9020
    T9233
    T9675
    T9872a
    T9961
    WWRec

This is the first step towards implementation of the linear types proposal
(https://github.com/ghc-proposals/ghc-proposals/pull/111).

It features

* A language extension -XLinearTypes
* Syntax for linear functions in the surface language
* Linearity checking in Core Lint, enabled with -dlinear-core-lint
* Core-to-core passes are mostly compatible with linearity
* Fields in a data type can be linear or unrestricted; linear fields
  have multiplicity-polymorphic constructors.
  If -XLinearTypes is disabled, the GADT syntax defaults to linear fields

The following items are not yet supported:

* a # m -> b syntax (only prefix FUN is supported for now)
* Full multiplicity inference (multiplicities are really only checked)
* Decent linearity error messages
* Linear let, where, and case expressions in the surface language
  (each of these currently introduce the unrestricted variant)
* Multiplicity-parametric fields
* Syntax for annotating lambda-bound or let-bound with a multiplicity
* Syntax for non-linear/multiple-field-multiplicity records
* Linear projections for records with a single linear field
* Linear pattern synonyms
* Multiplicity coercions (test LinearPolyType)

A high-level description can be found at
https://ghc.haskell.org/trac/ghc/wiki/LinearTypes/Implementation
Following the link above you will find a description of the changes made to Core.
This commit has been authored by

* Richard Eisenberg
* Krzysztof Gogolewski
* Matthew Pickering
* Arnaud Spiwack

With contributions from:

* Mark Barbone
* Alexander Vershilov

Updates haddock submodule.

The latter is apparently not supported by busybox.

It avoids using DynFlags in the Outputable instance of Clabel to check
assertions at pretty-printing time.

Just adding `{-# LANGUAGE BangPatterns #-}` makes the two other metrics
fluctuate by 13%.

See #16873.

This bit of documentation got outdated after commit
1fcede43

Some archives contain so called linker objects, with the affectionate
.lo suffic.  For example the musl libc.a will come in that form.  We
still want to load those objects, hence we should not discard them and
look for .lo as well.  Ultimately we might want to fix this proerly by
looking at the file magic.