GitLab

misc profiling cleanups

Numerous changes in the RTS to get GUM-4.06 working (currently works with
parfib-ish programs). Most changes are isolated in the rts/parallel dir.

rts/parallel/:
  The most important changes are a rewrite of the (un-)packing code (Pack.c)
  and changes in LAGA, GALA table operations (Global.c) expecially in
  rebuilding the tables during GC.

rts/:
  Minor changes in Schedule.c, GC.c (interface to par specific root marking
  and evacuation), and lots of additions to Sanity.c (surprise ;-)
  Main.c change for startup: I use a new function rts_evalNothing to
  start non-main-PEs in a PAR || SMP setup (RtsAPI.c)

includes/:
  Updated GranSim macros in PrimOps.h.

lib/std:
  Few changes in PrelHandle.c etc replacing ForeignObj by Addr in a PAR
  setup (we still don't support ForeignObjs or WeakPtrs in GUM).
  Typically use
    #define FILE_OBJECT	    Addr
  when dealing with files.

hslibs/lang/:
  Same as above (in Foreign(Obj).lhs, Weak.lhs, IOExts.lhs etc).

-- HWL

HEADS UP!!!

change the type of startupHaskell():

   void startupHaskell  ( int argc, char *argv[], void *init_root );

the extra parameter is a pointer to the initialisation function for
the root module in the program.  eg., Main.c now passes __init_Main for
this parameter.  It can be left as NULL if there is no root module.

This interface may need to be revised, since in some circumstances
there may be more than one "root module".

Sigbjorn: H/Direct will need some changes to stay in sync here.

Remove this file, it moved to ghc/rts.  (sigh, I thought I removed it
already, but apparently not).

This utterly gigantic commit is what I've been up to in background
mode in the last couple of months.  Originally the main goal
was to get rid of Con (staturated constant applications)
in the CoreExpr type, but one thing led to another, and I kept
postponing actually committing.   Sorry.

	Simon, 23 March 2000


I've tested it pretty thoroughly, but doubtless things will break.

Here are the highlights

* Con is gone; the CoreExpr type is simpler
* NoRepLits have gone
* Better usage info in interface files => less recompilation
* Result type signatures work
* CCall primop is tidied up
* Constant folding now done by Rules
* Lots of hackery in the simplifier
* Improvements in CPR and strictness analysis

Many bug fixes including

* Sergey's DoCon compiles OK; no loop in the strictness analyser
* Volker Wysk's programs don't crash the CPR analyser

I have not done much on measuring compilation times and binary sizes;
they could have got worse.  I think performance has got significantly
better, though, in most cases.


Removing the Con form of Core expressions
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
The big thing is that

  For every constructor C there are now *two* Ids:

	C is the constructor's *wrapper*. It evaluates and unboxes arguments
	before calling $wC.  It has a perfectly ordinary top-level defn
	in the module defining the data type.

	$wC is the constructor's *worker*.  It is like a primop that simply
	allocates and builds the constructor value.  Its arguments are the
	actual representation arguments of the constructor.
	Its type may be different to C, because:
		- useless dict args are dropped
		- strict args may be flattened

  For every primop P there is *one* Id, its (curried) Id

  Neither contructor worker Id nor the primop Id have a defminition anywhere.
  Instead they are saturated during the core-to-STG pass, and the code generator
  generates code for them directly. The STG language still has saturated
  primops and constructor applications.

* The Const type disappears, along with Const.lhs.  The literal part
  of Const.lhs reappears as Literal.lhs.  Much tidying up in here,
  to bring all the range checking into this one module.

* I got rid of NoRep literals entirely.  They just seem to be too much trouble.

* Because Con's don't exist any more, the funny C { args } syntax
  disappears from inteface files.


Parsing
~~~~~~~
* Result type signatures now work
	f :: Int -> Int = \x -> x
	-- The Int->Int is the type of f

	g x y :: Int = x+y
	-- The Int is the type of the result of (g x y)


Recompilation checking and make
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
* The .hi file for a modules is not touched if it doesn't change.  (It used to
  be touched regardless, forcing a chain of recompilations.)  The penalty for this
  is that we record exported things just as if they were mentioned in the body of
  the module.  And the penalty for that is that we may recompile a module when
  the only things that have changed are the things it is passing on without using.
  But it seems like a good trade.

* -recomp is on by default

Foreign declarations
~~~~~~~~~~~~~~~~~~~~
* If you say
	foreign export zoo :: Int -> IO Int
  then you get a C produre called 'zoo', not 'zzoo' as before.
  I've also added a check that complains if you export (or import) a C
  procedure whose name isn't legal C.


Code generation and labels
~~~~~~~~~~~~~~~~~~~~~~~~~~
* Now that constructor workers and wrappers have distinct names, there's
  no need to have a Foo_static_closure and a Foo_closure for constructor Foo.
  I nuked the entire StaticClosure story.  This has effects in some of
  the RTS headers (i.e. s/static_closure/closure/g)


Rules, constant folding
~~~~~~~~~~~~~~~~~~~~~~~
* Constant folding becomes just another rewrite rule, attached to the Id for the
  PrimOp.   To achieve this, there's a new form of Rule, a BuiltinRule (see CoreSyn.lhs).
  The prelude rules are in prelude/PrelRules.lhs, while simplCore/ConFold.lhs has gone.

* Appending of constant strings now works, using fold/build fusion, plus
  the rewrite rule
	unpack "foo" c (unpack "baz" c n)  =  unpack "foobaz" c n
  Implemented in PrelRules.lhs

* The CCall primop is tidied up quite a bit.  There is now a data type CCall,
  defined in PrimOp, that packages up the info needed for a particular CCall.
  There is a new Id for each new ccall, with an big "occurrence name"
	{__ccall "foo" gc Int# -> Int#}
  In interface files, this is parsed as a single Id, which is what it is, really.

Miscellaneous
~~~~~~~~~~~~~
* There were numerous places where the host compiler's
  minInt/maxInt was being used as the target machine's minInt/maxInt.
  I nuked all of these; everything is localised to inIntRange and inWordRange,
  in Literal.lhs

* Desugaring record updates was broken: it didn't generate correct matches when
  used withe records with fancy unboxing etc.  It now uses matchWrapper.

* Significant tidying up in codeGen/SMRep.lhs

* Add __word, __word64, __int64 terminals to signal the obvious types
  in interface files.  Add the ability to print word values in hex into
  C code.

* PrimOp.lhs is no longer part of a loop.  Remove PrimOp.hi-boot*


Types
~~~~~
* isProductTyCon no longer returns False for recursive products, nor
  for unboxed products; you have to test for these separately.
  There's no reason not to do CPR for recursive product types, for example.
  Ditto splitProductType_maybe.

Simplification
~~~~~~~~~~~~~~~
* New -fno-case-of-case flag for the simplifier.  We use this in the first run
  of the simplifier, where it helps to stop messing up expressions that
  the (subsequent) full laziness pass would otherwise find float out.
  It's much more effective than previous half-baked hacks in inlining.

  Actually, it turned out that there were three places in Simplify.lhs that
  needed to know use this flag.

* Make the float-in pass push duplicatable bindings into the branches of
  a case expression, in the hope that we never have to allocate them.
  (see FloatIn.sepBindsByDropPoint)

* Arrange that top-level bottoming Ids get a NOINLINE pragma
  This reduced gratuitous inlining of error messages.
  But arrange that such things still get w/w'd.

* Arrange that a strict argument position is regarded as an 'interesting'
  context, so that if we see
	foldr k z (g x)
  then we'll be inclined to inline g; this can expose a build.

* There was a missing case in CoreUtils.exprEtaExpandArity that meant
  we were missing some obvious cases for eta expansion
  Also improve the code when handling applications.

* Make record selectors (identifiable by their IdFlavour) into "cheap" operations.
	  [The change is a 2-liner in CoreUtils.exprIsCheap]
  This means that record selection may be inlined into function bodies, which
  greatly improves the arities of overloaded functions.

* Make a cleaner job of inlining "lone variables".  There was some distributed
  cunning, but I've centralised it all now in SimplUtils.analyseCont, which
  analyses the context of a call to decide whether it is "interesting".

* Don't specialise very small functions in Specialise.specDefn
  It's better to inline it.  Rather like the worker/wrapper case.

* Be just a little more aggressive when floating out of let rhss.
  See comments with Simplify.wantToExpose
  A small change with an occasional big effect.

* Make the inline-size computation think that
	case x of I# x -> ...
  is *free*.


CPR analysis
~~~~~~~~~~~~
* Fix what was essentially a bug in CPR analysis.  Consider

	letrec f x = let g y = let ... in f e1
		     in
		     if ... then (a,b) else g x

  g has the CPR property if f does; so when generating the final annotated
  RHS for f, we must use an envt in which f is bound to its final abstract
  value.  This wasn't happening.  Instead, f was given the CPR tag but g
  wasn't; but of course the w/w pass gives rotten results in that case!!
  (Because f's CPR-ness relied on g's.)

  On they way I tidied up the code in CprAnalyse.  It's quite a bit shorter.

  The fact that some data constructors return a constructed product shows
  up in their CPR info (MkId.mkDataConId) not in CprAnalyse.lhs



Strictness analysis and worker/wrapper
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
* BIG THING: pass in the demand to StrictAnal.saExpr.  This affects situations
  like
	f (let x = e1 in (x,x))
  where f turns out to have strictness u(SS), say.  In this case we can
  mark x as demanded, and use a case expression for it.

  The situation before is that we didn't "know" that there is the u(SS)
  demand on the argument, so we simply computed that the body of the let
  expression is lazy in x, and marked x as lazily-demanded.  Then even after
  f was w/w'd we got

	let x = e1 in case (x,x) of (a,b) -> $wf a b

  and hence

	let x = e1 in $wf a b

  I found a much more complicated situation in spectral/sphere/Main.shade,
  which improved quite a bit with this change.

* Moved the StrictnessInfo type from IdInfo to Demand.  It's the logical
  place for it, and helps avoid module loops

* Do worker/wrapper for coerces even if the arity is zero.  Thus:
	stdout = coerce Handle (..blurg..)
  ==>
	wibble = (...blurg...)
	stdout = coerce Handle wibble
  This is good because I found places where we were saying
	case coerce t stdout of { MVar a ->
	...
	case coerce t stdout of { MVar b ->
	...
  and the redundant case wasn't getting eliminated because of the coerce.

Zap outdated ifdeffery.

Adding an alternative to the "delay" system used for
threads that are waiting for time to pass.

This works on a target time basis, eliminating the
need to use the ticky style counter.

It is only enabled under:

#if defined(INTERPRETER) && !defined(HAVE_SETITIMER)

Second attack at supporting threads inside STG Hugs.
We now support most of the concurrency primitives.

Also a wibble in Evaluator.c, letting Hugs compile.

More random cleanups.

Random cleanup: rename the TSO->whatNext field to what_next, to be a
bit more consistent.

Add the readBlahOffAddr suite of primitives.  The previous method of
using indexStuffOffAddr didn't enforce proper ordering in the I/O
monad.

The indexBlahOffAddr primops may go away in the future if/when we
figure out how to make unsafePerformIO into a no-op at the back end.

Add global_link field to the TSO structure.  Forgotten in yesterday's
commit.

Use Sp instead of the global init_stack for the stack pointer during
module initialisation.  This should cut down the code size for module
initialisation blocks.

optimise SaveThreadState/LoadThreadState slightly by caching
CurrentTSO in a local variable.

Simplified prog_* externs a little

Remove unsafeThawByteArray# primop (which was a no-op), and use
unsafeCoerce# instead.

Move Prelude.h from ghc/includes into ghc/rts.  It's essentially a
list of the things from the Prelude that the RTS depends on, and isn't
relevant to STG code.  Furthermore, this helps to clarify which parts
of the RTS depend on things from the Prelude (by an explicity
#include).

remove unused gunk.

Nuke a substantial number of no-longer-supported options.

First stab at reviving TREX - just fixing some header wibbles.  Also one small
fix to get implicit parameters working (`textOf' needed to be taught that
IPCELLs are also OK).  Moved #defines for TREX, IPARAM, etc, into options.h
(in their previous location at the end of prelude.h, they were being defined
too late).

Compilation cleanups:
* Add some prototypes to header files, to avoid compilation warnings.
* Remove irrelevant #ifdeffery (#if NPLUSK, + various others)

improving the synatax and semantics of the privileged import

 Typical use might be:

  import Prelude
  import privileged Prelude ( IORef , unsafePerformIO )

 Which means please ignore the export that comes with Prelude,
 and let me at compiler internal magic operations, IORef and
 unsafePerformIO (both are later exported by IOExt)

I've also updated the stdlib files to use this (hugs only :-).

- generalise the per-module initialisation stubs so that we use it
  in normal (non-profiled) code too.  The initialisation stubs are
  now called '__init_<module>' rather than '_reg<module>'.

- Register foreign exported functions as stable pointers in the
  initialisation code for the module.  This fixes the foreign export
  problems reported by several people.

- remove the concept of "module groups" from the profiling subsystem.

- change the profiling semantics slightly; it should be unnecessary
  to use '-caf-all' to get reasonable profiles now.

Adding prelude changes require for the new libs, include IOExts.
Reintroducing ptr equality into HugsSTG.

Change the behaviour of the cost-centre stack profiler for recursive
sccs.  Before, we used to remove the old copy of the CC from the stack
and push the new one on.  Now, we record back-edges in the graph when
a recursive scc is detected.

This should give saner-looking cost-centre stacks for heavily
recursive code.  Indeed, it reduces the number of stacks in the
profile for some examples considerably.

Many changes to improve the quality and correctness of generated code,
both for x86 and all-platforms.  The intent is that the x86 NCG will
now be good enough for general use.

-- Add an almost-trivial Stix (generic) peephole optimiser, whose sole
   purpose is elide assignments to temporaries used only once, in the
   very next tree.  This generates substantially better code for
   conditionals on all platforms.  Enhance Stix constant folding to
   take advantage of the inlining.

   The inlining presents subsequent insn selection phases with more
   complex trees than would have previously been used to.  This has
   shown up several bugs in the x86 insn selectors, now fixed.
   (assumptions that data size is Word, when could be Byte,
    assumptions that an operand will always be in a temp reg, etc)

-- x86: Use the FLDZ and FLD1 insns.

-- x86: spill FP registers with 80-bit loads/stores so that
   Intel's extra 16 bits of accuracy are not lost.  If this isn't
   done, FP spills are not suitably transparent.  Increase the
   number of spill words available to 2048.

-- x86: give the register allocator more flexibility in choosing
   spill temporaries.

-- x86, RegAllocInfo.regUsage: fix error for GST, and rewrite to
   make it clearer.

-- Correctly track movements in the C stack pointer, and generate
   correct spill code for archs which spill against the stack pointer
   even when the stack pointer moves.  Redo the x86 ccall mechanism
   to push args on the C stack in the normal way.  Rather than have
   the spiller have to analyse code sequences to determine the current
   stack offset, the insn selectors communicate the current offset
   whenever it changes by inserting a DELTA pseudo-insn.  Then the
   spiller only has to spot DELTAs.

   This means having a new native-code-generator monad (Stix.NatM)
   which carries both a UniqSupply and the current stack offset.

-- Remove the asmPar/asmSeq ways of grouping insns together.
   In the presence of fixed registers, it is hard to demonstrate
   that insn selectors using asmPar always give correct code, and
   the extra complication doesn't help any.

   Also, directly construct code sequences using tree-based ordered
   lists (utils/OrdList.lhs) for linear-time appends, rather than
   the bizarrely complex method using fns and fn composition.

-- Inline some hcats in printing of x86 address modes.

-- Document more of the hidden assumptions which insn selection relies
   on, particular wrt addressing modes.

Nuke PTR_ON_HEAP and all associated ifdeffery; we always need it.

Double the number of RESERVED_C_STACK_BYTES so as to give the
native code generator up to 508 spill slots.

Implement the HP_CHK_GEN macro.  As a result, teach mkNativeHdr et al
about R9 and R10.

Added a bunch of #defines for {SIZEOF,ALIGNMENT}_{INT,WORD}{8,16,32,64}.
All this is a real hack: The include files need a more thorough restructuring.

Added note about necessary synching with MachDeps.h and fixed a small typo

ARR_HDR_SIZE --> ARR_WORDS_HDR_SIZE, and derived quantities in
Constants.h, Constants.lhs et al are similarly renamed.

new constant ARR_PTRS_HDR_SIZE, with corresponding derivatives.

Fix bug in async exception handling: the target TSO may have been
relocated as a result of a stack overflow.

Introduce a new StgTSOWhatNext value "ThreadRelocated", which
indicates that this TSO has moved, and the new location is in the link
field.  The garbage collector shorts these out just like indirections.

We have to check for relocated TSOs in killThread# (and any other
primops which take a ThreadId# as an argument - there aren't any at
present).

- context switch after a `par`
- remove some #if 0'd code

non-functional changes that were sitting in my tree.

Merged GUM-4-04 branch into the main trunk. In particular merged GUM and
SMP code. Most of the GranSim code in GUM-4-04 still has to be carried over.

- remove AllBlocked scheduler return code.  Nobody owned up to having
  created it or even knowing what it was there for.

- clean up fatal error condition handling somewhat.  The process
  exit code from a GHC program now indicates the kind of failure
  for certain kinds of exit:

	general internal RTS error	254
	program deadlocked		253
	program interrupted (ctrl-C)	252
	heap overflow			251
	main thread killed		250

  and we leave exit codes 1-199 for the user (as is traditional at MS,
  200-249 are reserved for future expansion, and may contain
  undocumented extensions :-)

Add 'par' and sparking support to the SMP implementation.

Remove _fast from macro name gcdIntegerIntzh_fast.

Change NUM_TUPLES to 37, to match GHC.