GitLab

I'd been meaning to do this for some time, but finally got around to it
due to the overflowing literals warning. With that enabled, we were
getting a warning for
    -128 :: Int8
as that is parsed as
    negate (fromInteger 128)
which just happens to do the right thing, as
    negate (fromInteger 128) = negate (-128) = -128

Makes it look less likely that people will confuse what it is for
(e.g. #8104).

They now go through log_action. The existing severities all used
printDoc, which always adds a trailing newline, which we don't
want for the GHCi messages. I therefore added a new severity
SevInteractive, which doesn't add a newline.

Previously, we did ./configure time checks to see if 'GNU ld' supported
certain options. If it does, we bake those options into the link step.
See Trac #5240.

Unfortunately, the linker we use at runtime can change for several
reasons. One is that the user specifies -pgml 'foo'. The other is if
/usr/bin/ld or whatnot changes from when GHC was built.  Those options
mentioned earlier are specific to GNU ld, but many systems support GNU
gold too. This is Trac #6063.

So we need to check at runtime what linker we're using. This is actually
a little bit complicated because we normally use the C compiler as our
linker. Windows and OS X are also special here.

Finally, this patch also unconditionally gives '--hash-size=31' and
'--reduce-memory-overheads' to the system linker if it's GNU ld. These
options have been supported for 8+ years from what I can see, and there
are probably a lot of other reasons why GHC would not work with such an
ancient binutils, all things considered.

See Note [Run-time linker info] in SysTools for details. There are
plenty of comments as well in the surrounding code.
Signed-off-by: Austin Seipp <aseipp@pobox.com>

This major patch implements the cardinality analysis described
in our paper "Higher order cardinality analysis". It is joint
work with Ilya Sergey and Dimitrios Vytiniotis.

The basic is augment the absence-analysis part of the demand
analyser so that it can tell when something is used
	 never
	 at most once
 	 some other way

The "at most once" information is used
    a) to enable transformations, and
       in particular to identify one-shot lambdas
    b) to allow updates on thunks to be omitted.

There are two new flags, mainly there so you can do performance
comparisons:
    -fkill-absence   stops GHC doing absence analysis at all
    -fkill-one-shot  stops GHC spotting one-shot lambdas
                     and single-entry thunks

The big changes are:

* The Demand type is substantially refactored.  In particular
  the UseDmd is factored as follows
      data UseDmd
        = UCall Count UseDmd
        | UProd [MaybeUsed]
        | UHead
        | Used

      data MaybeUsed = Abs | Use Count UseDmd

      data Count = One | Many

  Notice that UCall recurses straight to UseDmd, whereas
  UProd goes via MaybeUsed.

  The "Count" embodies the "at most once" or "many" idea.

* The demand analyser itself was refactored a lot

* The previously ad-hoc stuff in the occurrence analyser for foldr and
  build goes away entirely.  Before if we had build (\cn -> ...x... )
  then the "\cn" was hackily made one-shot (by spotting 'build' as
  special.  That's essential to allow x to be inlined.  Now the
  occurrence analyser propagates info gotten from 'build's stricness
  signature (so build isn't special); and that strictness sig is
  in turn derived entirely automatically.  Much nicer!

* The ticky stuff is improved to count single-entry thunks separately.

One shortcoming is that there is no DEBUG way to spot if an
allegedly-single-entry thunk is acually entered more than once.  It
would not be hard to generate a bit of code to check for this, and it
would be reassuring.  But it's fiddly and I have not done it.

Despite all this fuss, the performance numbers are rather under-whelming.
See the paper for more discussion.

       nucleic2          -0.8%    -10.9%      0.10      0.10     +0.0%
         sphere          -0.7%     -1.5%      0.08      0.08     +0.0%
--------------------------------------------------------------------------------
            Min          -4.7%    -10.9%     -9.3%     -9.3%    -50.0%
            Max          -0.4%     +0.5%     +2.2%     +2.3%     +7.4%
 Geometric Mean          -0.8%     -0.2%     -1.3%     -1.3%     -1.8%

I don't quite know how much credence to place in the runtime changes,
but movement seems generally in the right direction.

It doesn't work

Now these are always added by the run<blah> functions in SysTools, so
we never miss any out.  Several cleanups resulted.

When GHCi makes temporary DLLs, those also need to be linked against
the right RTS, or we won't be able to load them.

There's now an internal -dll-split flag, which we use to tell GHC how
the GHC package is split into 2 separate DLLs. This is used by
Packages.isDllName to determine whether a call is within the same
DLL, or whether it is a call to another DLL.

The flag shows no real impact on nofib benchmarks and GHC itself,
which is expected given the small number of strict but not already
unpacked fields in the source of these programs. However, the flag
allows us to omit most of the UNPACK pragmas that are so common in
source code today.

We now just pass the filename as an argument

We now just pass the output filename as an argument instead

It doesn't work with --make yet. Trac #7864 filed.

This is always "YES".

It was checking to see whether the hscTarget is HscC or HscLlvm, and
changing it to HscC otherwise. But this meant that as well as rejecting
HscAsm, it also rejected HscInterpreted and HscNothing.

Now we just check whether it's HscAsm.

size_expr now ignores RealWorld lambdas, arguments, and applications.

Worker-wrapper previously removed all lambdas from a function, if they
were all unused. Removing *all* value lambdas is no longer
allowed. Instead (\_ -> E) will become (\_void -> E), where it used to
become E. The previous behavior can be recovered via the new
-ffun-to-thunk flag.

Nofib notables:

----------------------------------------------------------------
        Program               O2          O2 newly ignoring RealWorld
                                          and not turning function
                                          closures into thunks
----------------------------------------------------------------

 Allocations

  comp_lab_zift            333090392%           -5.0%
reverse-complem            155188304%           -3.2%

        rewrite             15380888%           +4.0%
         boyer2              3901064%           +7.5%

rewrite previously benefited from fortunate LoopBreaker choice that is
now disrupted.

A function in boyer2 goes from $wonewayunify1 size 700 to size 650,
thus gets inlined into rewritelemmas, thus exposing a parameter
scrutinisation, thus allowing SpecConstr, which unfortunately involves
reboxing.

Run Time

 fannkuch-redux                 7.89%          -15.9%

            hpg                 0.25%           +5.6%
           wang                 0.21%           +5.8%

/shrug

  * the new StgCmmArgRep module breaks a dependency cycle; I also
    untabified it, but made no real changes

  * updated the documentation in the wiki and change the user guide to
    point there

  * moved the allocation enters for ticky and CCS to after the heap check

    * I left LDV where it was, which was before the heap check at least
      once, since I have no idea what it is

  * standardized all (active?) ticky alloc totals to bytes

  * in order to avoid double counting StgCmmLayout.adjustHpBackwards
    no longer bumps ALLOC_HEAP_ctr

  * I resurrected the SLOW_CALL counters

    * the new module StgCmmArgRep breaks cyclic dependency between
      Layout and Ticky (which the SLOW_CALL counters cause)

    * renamed them SLOW_CALL_fast_<pattern> and VERY_SLOW_CALL

  * added ALLOC_RTS_ctr and _tot ticky counters

    * eg allocation by Storage.c:allocate or a BUILD_PAP in stg_ap_*_info

    * resurrected ticky counters for ALLOC_THK, ALLOC_PAP, and
      ALLOC_PRIM

    * added -ticky and -DTICKY_TICKY in ways.mk for debug ways

  * added a ticky counter for total LNE entries

  * new flags for ticky: -ticky-allocd -ticky-dyn-thunk -ticky-LNE

    * all off by default

    * -ticky-allocd: tracks allocation *of* closure in addition to
       allocation *by* that closure

    * -ticky-dyn-thunk tracks dynamic thunks as if they were functions

    * -ticky-LNE tracks LNEs as if they were functions

  * updated the ticky report format, including making the argument
    categories (more?) accurate again

  * the printed name for things in the report include the unique of
    their ticky parent as well as if they are not top-level

ForceSpecConstr will now only specialise recursive types a finite number of times.
There is a new option -fspec-constr-recursive, with a default value of 3.

There's no point splitting objects when we're going to be dynamically
linking. Plus it breaks compilation on OSX x86.

Without it, when linking the split objects for Language.Haskell.TH.Syntax,
the commandline was too long when listing all the files directly.

LLVM supports PIC on ARM from what I can see.
Signed-off-by: Austin Seipp <aseipp@pobox.com>

We were previously enabling Opt_PIC on certain OSes namely Windows,
Darwin and Linux, but it should always be enabled for the dyn way
because it makes no sense to try to generate non-PIC dynamic
libraries, which is usually impossible.

On OpenBSD and NetBSD, we were accidentally passing "-optl-pthread" to
the gcc to build dynamic libraries because of a bitrotted hack for
those OSes, which resulted in a weird situation where every dynamic
library is named "ptl-pthread" and placed at "$(TOP)".

The hack in question were to work around a linkage problem with
libHSffi, but we no longer build it and just use libffi these days so
the hack can safely be removed.

There were some cases where we weren't unsetting it when turning the
Dyn way on.

In particular, this means that GHCi will use DLLs, rather than loading
object files itself.

This is a slightly refined version of a patch by shachaf,
done by Krzysztof Gogolewski <krz.gogolewski@gmail.com>.

I'm not sure why it caused problems before, but it's fine now.

We don't need them, but GHC API users might.