GitLab

Summary:
In this example we ended up with some code that was only reachable via
an info table, because a branch had been optimised away by the native
code generator.  The register allocator then got confused because it
was only considering the first block of the proc to be an entry point,
when actually any of the info tables are entry points.

Test Plan: validate

Reviewers: simonpj, austin

Subscribers: simonmar, relrod, carter

Differential Revision: https://phabricator.haskell.org/D88

A previous fix to this was wrong: f5879acd
and left some unreachable code behind.  So rather than try to be clever and
do this at the same time as the strongly-connected-component analysis, I'm
doing a separate reachability pass first.

In some cases, the layout of the LANGUAGE/OPTIONS_GHC lines has been
reorganized, while following the convention, to

- place `{-# LANGUAGE #-}` pragmas at the top of the source file, before
  any `{-# OPTIONS_GHC #-}`-lines.

- Moreover, if the list of language extensions fit into a single
  `{-# LANGUAGE ... -#}`-line (shorter than 80 characters), keep it on one
  line. Otherwise split into `{-# LANGUAGE ... -#}`-lines for each
  individual language extension. In both cases, try to keep the
  enumeration alphabetically ordered.
  (The latter layout is preferable as it's more diff-friendly)

While at it, this also replaces obsolete `{-# OPTIONS ... #-}` pragma
occurences by `{-# OPTIONS_GHC ... #-}` pragmas.

This checks that all the required extensions are enabled for the
inferred type signature.

Updates binary and vector submodules.

The problem with unreachable code is that it might refer to undefined
registers.  This happens accidentally: a block can be orphaned by an
optimisation, for example when the result of a comparsion becomes
known.

The register allocator panics when it finds an undefined register,
because they shouldn't occur in generated code.  So we need to also
discard unreachable code to prevent this panic being triggered by
optimisations.

The register alloator already does a strongly-connected component
analysis, so it ought to be easy to make it discard unreachable code
as part of that traversal.  It turns out that we need a different
variant of the scc algorithm to do that (see Digraph), however the new
variant also generates slightly better code by putting the blocks
within a loop in a better order for register allocation.

This removes the OldCmm data type and the CmmCvt pass that converts
new Cmm to OldCmm.  The backends (NCGs, LLVM and C) have all been
converted to consume new Cmm.

The main difference between the two data types is that conditional
branches in new Cmm have both true/false successors, whereas in OldCmm
the false case was a fallthrough.  To generate slightly better code we
occasionally need to invert a conditional to ensure that the
branch-not-taken becomes a fallthrough; this was previously done in
CmmCvt, and it is now done in CmmContFlowOpt.

We could go further and use the Hoopl Block representation for native
code, which would mean that we could use Hoopl's postorderDfs and
analyses for native code, but for now I've left it as is, using the
old ListGraph representation for native code.

All Cmm procedures now include the set of global registers that are live on
procedure entry, i.e., the global registers used to pass arguments to the
procedure. Only global registers that are use to pass arguments are included in
this list.

Proc-point splitting is only required by backends that do not support
having proc-points within a code block (that is, everything except the
native backend, i.e. LLVM and C).

Not doing proc-point splitting saves some compilation time, and might
produce slightly better code in some cases.

We can now get the Platform from the DynFlags inside an SDoc, so we
no longer need to pass the Platform in.

And some knock-on changes

   CmmTop -> CmmDecl
   CmmPgm -> CmmGroup

There's now a variant of the Outputable class that knows what
platform we're targetting:

class PlatformOutputable a where
    pprPlatform :: Platform -> a -> SDoc
    pprPlatformPrec :: Platform -> Rational -> a -> SDoc

and various instances have had to be converted to use that class,
and we pass Platform around accordingly.

I introduced this to support explicitly recording the info table label
in RawCmm for another patch I am working on, but it turned out to lead
to significant simplification in those parts of the compiler that
consume RawCmm.

Now, instead of lots of tests for null [CmmStatic] we have a simple
test of a Maybe, and have reduced the number of guys that need to know
how to convert entry->info labels by a TON. There are only 3 callers
of that function now!

I observed that the [CmmStatics] within CmmData uses the list in a very stylised way.
The first item in the list is almost invariably a CmmDataLabel. Many parts of the
compiler pattern match on this list and fail if this is not true.

This patch makes the invariant explicit by introducing a structured type CmmStatics
that holds the label and the list of remaining [CmmStatic].

There is one wrinkle: the x86 backend sometimes wants to output an alignment directive just
before the label. However, this can be easily fixed up by parameterising the native codegen
over the type of CmmStatics (though the GenCmmTop parameterisation) and using a pair
(Alignment, CmmStatics) there instead.

As a result, I think we will be able to remove CmmAlign and CmmDataLabel from the CmmStatic
data type, thus nuking a lot of code and failing pattern matches. This change will come as part
of my next patch.

This changes the new code generator to make use of the Hoopl package
for dataflow analysis.  Hoopl is a new boot package, and is maintained
in a separate upstream git repository (as usual, GHC has its own
lagging darcs mirror in http://darcs.haskell.org/packages/hoopl).

During this merge I squashed recent history into one patch.  I tried
to rebase, but the history had some internal conflicts of its own
which made rebase extremely confusing, so I gave up. The history I
squashed was:

  - Update new codegen to work with latest Hoopl
  - Add some notes on new code gen to cmm-notes
  - Enable Hoopl lag package.
  - Add SPJ note to cmm-notes
  - Improve GC calls on new code generator.

Work in this branch was done by:
   - Milan Straka <fox@ucw.cz>
   - John Dias <dias@cs.tufts.edu>
   - David Terei <davidterei@gmail.com>

Edward Z. Yang <ezyang@mit.edu> merged in further changes from GHC HEAD
and fixed a few bugs.

This major patch implements the new OutsideIn constraint solving
algorithm in the typecheker, following our JFP paper "Modular type
inference with local assumptions".  

Done with major help from Dimitrios Vytiniotis and Brent Yorgey.

 computeLiveness requires the SCCs of blocks to be in reverse dependent
 order, and if they're not it was silently giving bad liveness info, 
 yielding a bad allocation.

 Now it complains, loudly.

 * I've pushed the SPILL and RELOAD instrs down into the
   LiveInstr type to make them easier to work with. 

 * When the graph allocator does a spill cycle it now just
   re-annotates the LiveCmmTops instead of converting them 
   to NatCmmTops and back. 

 * This saves working out the SCCS again, and avoids rewriting
   the SPILL and RELOAD meta instructions into real machine
   instructions.

 * The old Reg type is now split into VirtualReg and RealReg.
 * For the graph coloring allocator, the type of the register graph
   is now (Graph VirtualReg RegClass RealReg), which shows that it colors
   in nodes representing virtual regs with colors representing real regs.
   (as was intended)  
 * RealReg contains two contructors, RealRegSingle and RealRegPair,
   where RealRegPair is used to represent a SPARC double reg 
   constructed from two single precision FP regs. 
 * On SPARC we can now allocate double regs into an arbitrary register
   pair, instead of reserving some reg ranges to only hold float/double values. 

  - nativeGen/Instruction defines a type class for a generic
    instruction set. Each of the instruction sets we have, 
    X86, PPC and SPARC are instances of it.
  
  - The register alloctors use this type class when they need
    info about a certain register or instruction, such as
    regUsage, mkSpillInstr, mkJumpInstr, patchRegs..
  
  - nativeGen/Platform defines some data types enumerating
    the architectures and operating systems supported by the 
    native code generator.
  
  - DynFlags now keeps track of the current build platform, and 
    the PositionIndependentCode module uses this to decide what
    to do instead of relying of #ifdefs.
  
  - It's not totally retargetable yet. Some info info about the
    build target is still hardwired, but I've tried to contain
    most of it to a single module, TargetRegs.
  
  - Moved the SPILL and RELOAD instructions into LiveInstr.
  
  - Reg and RegClass now have their own modules, and are shared
    across all architectures.

o Fixed bug that emitted the copy-in code for closure entry
  in the wrong place -- at the initialization of the closure.
o Refactored some of the closure entry code.
o Added code to check that no LocalRegs are live-in to a procedure
   -- trip up some buggy programs earlier
o Fixed environment bindings for thunks
   -- we weren't (re)binding the free variables in a thunk
o Fixed a bug in proc-point splitting that dropped some updates
  to the entry block in a procedure.
o Fixed improper calls to code that generates CmmLit's for strings
o New invariant on cg_loc in CgIdInfo: the expression is always tagged
o Code to load free vars on entry to a thunk was (wrongly) placed before
  the heap check.
o Some of the StgCmm code was redundantly passing around Id's
  along with CgIdInfo's; no more.
o Initialize the LocalReg's that point to a closure before allocating and
  initializing the closure itself -- otherwise, we have problems with
  recursive closure bindings
o BlockEnv and BlockSet types are now abstract.
o Update frames:
  - push arguments in Old call area
  - keep track of the return sp in the FCode monad
  - keep the return sp in every call, tail call, and return
      (because it might be different at different call sites,
       e.g. tail calls to the gc after a heap check are performed
            before pushing the update frame)
  - set the sp appropriately on returns and tail calls
o Reduce call, tail call, and return to a single LastCall node
o Added slow entry code, using different calling conventions on entry and tail call
o More fixes to the calling convention code.
  The tricky stuff is all about the closure environment: it must be passed in R1,
  but in non-closures, there is no such argument, so we can't treat all arguments
  the same way: the closure environment is special. Maybe the right step forward
  would be to define a different calling convention for closure arguments.
o Let-no-escapes need to be emitted out-of-line -- otherwise, we drop code.
o Respect RTS requirement of word alignment for pointers
  My stack allocation can pack sub-word values into a single word on the stack,
  but it wasn't requiring word-alignment for pointers. It does now,
  by word-aligning both pointer registers and call areas.
o CmmLint was over-aggresively ruling out non-word-aligned memory references,
  which may be kosher now that we can spill small values into a single word.
o Wrong label order on a conditional branch when compiling switches.
o void args weren't dropped in many cases.
  To help prevent this kind of mistake, I defined a NonVoid wrapper,
  which I'm applying only to Id's for now, although there are probably
  other good candidates.
o A little code refactoring: separate modules for procpoint analysis splitting, 
  stack layout, and building infotables.
o Stack limit check: insert along with the heap limit check, using a symbolic
  constant (a special CmmLit), then replace it when the stack layout is known.
o Removed last node: MidAddToContext 
o Adding block id as a literal: means that the lowering of the calling conventions
  no longer has to produce labels early, which was inhibiting common-block elimination.
  Will also make it easier for the non-procpoint-splitting path.
o Info tables: don't try to describe the update frame!
o Over aggressive use of NonVoid!!!!
  Don't drop the non-void args before setting the type of the closure!!!
o Sanity checking:
  Added a pass to stub dead dead slots on the stack
  (only ~10 lines with the dataflow framework)
o More sanity checking:
  Check that incoming pointer arguments are non-stubbed.
  Note: these checks are still subject to dead-code removal, but they should
  still be quite helpful.
o Better sanity checking: why stop at function arguments?
  Instead, in mkAssign, check that _any_ assignment to a pointer type is non-null
  -- the sooner the crash, the easier it is to debug.
  Still need to add the debugging flag to turn these checks on explicitly.
o Fixed yet another calling convention bug.
  This time, the calls to the GC were wrong. I've added a new convention
  for GC calls and invoked it where appropriate.
  We should really straighten out the calling convention stuff:
    some of the code (and documentation) is spread across the compiler,
    and there's some magical use of the node register that should really
    be handled (not avoided) by calling conventions.
o Switch bug: the arms in mkCmmLitSwitch weren't returning to a single join point.
o Environment shadowing problem in Stg->Cmm:
  When a closure f is bound at the top-level, we should not bind f to the
  node register on entry to the closure.
  Why? Because if the body of f contains a let-bound closure g that refers
  to f, we want to make sure that it refers to the static closure for f.
  Normally, this would all be fine, because when we compile a closure,
  we rebind free variables in the environment. But f doesn't look like
  a free variable because it's a static value. So, the binding for f
  remains in the environment when we compile g, inconveniently referring
  to the wrong thing.
  Now, I bind the variable in the local environment only if the closure is not
  bound at the top level. It's still okay to make assumptions about the
  node holding the closure environment; we just won't find the binding
  in the environment, so code that names the closure will now directly
  get the label of the static closure, not the node register holding a
  pointer to the static closure.
o Don't generate bogus Cmm code containing SRTs during the STG -> Cmm pass!
  The tables made reference to some labels that don't exist when we compute and
  generate the tables in the back end.
o Safe foreign calls need some special treatment (at least until we have the integrated
  codegen). In particular:
  o they need info tables
  o they are not procpoints -- the successor had better be in the same procedure
  o we cannot (yet) implement the calling conventions early, which means we have
    to carry the calling-conv info all the way to the end
o We weren't following the old convention when registering a module.
  Now, we use update frames to push any new modules that have to be registered
  and enter the youngest one on the stack.
  We also use the update frame machinery to specify that the return should pop
  the return address off the stack.
o At each safe foreign call, an infotable must be at the bottom of the stack,
  and the TSO->sp must point to it.
o More problems with void args in a direct call to a function:
  We were checking the args (minus voids) to check whether the call was saturated,
  which caused problems when the function really wasn't saturated because it
  took an extra void argument.
o Forgot to distinguish integer != from floating != during Stg->Cmm
o Updating slotEnv and areaMap to include safe foreign calls
  The dataflow analyses that produce the slotEnv and areaMap give
  results for each basic block, but we also need the results for
  a safe foreign call, which is a middle node.
  After running the dataflow analysis, we have another pass that
  updates the results to includ any safe foreign calls.
o Added a static flag for the debugging technique that inserts
  instructions to stub dead slots on the stack and crashes when
  a stubbed value is loaded into a pointer-typed LocalReg.
o C back end expects to see return continuations before their call sites.
  Sorted the flowgraphs appropriately after splitting.
o PrimOp calling conventions are special -- unlimited registers, no stack
  Yet another calling convention...
o More void value problems: if the RHS of a case arm is a void-typed variable,
  don't try to return it.
o When calling some primOp, they may allocate memory; if so, we need to
  do a heap check when we return from the call.