GitLab

To explicitly choose whether you want an unregisterised build you now
need to use the "--enable-unregisterised"/"--disable-unregisterised"
configure flags.

This is a bit odd by itself, but it's a stepping stone on the way to
putting "target unregisterised" into the settings file.

All the flags that 'ways' imply are now dynamic

By using Haskell's debugIsOn rather than CPP's "#ifdef DEBUG", we
don't need to kludge things to keep the warning checker happy etc.

We now carry around with CmmJump statements a list of
the STG registers that are live at that jump site.
This is used by the LLVM backend so it can avoid
unnesecarily passing around dead registers, improving
perfromance. This gives us the framework to finally
fix trac #4308.

We only use it for "compiler" sources, i.e. not for libraries.
Many modules have a -fno-warn-tabs kludge for now.

Simon Peyton Jones authored Aug 22, 2011 and Simon Marlow committed Aug 25, 2011

* Move CgRep (private to old codgen) from SMRep to ClosureInfo
* Avoid using CgRep in new codegen
* Move SMRep and Bitmap from codeGen/ to cmm/

This changes the new code generator to make use of the Hoopl package
for dataflow analysis.  Hoopl is a new boot package, and is maintained
in a separate upstream git repository (as usual, GHC has its own
lagging darcs mirror in http://darcs.haskell.org/packages/hoopl).

During this merge I squashed recent history into one patch.  I tried
to rebase, but the history had some internal conflicts of its own
which made rebase extremely confusing, so I gave up. The history I
squashed was:

  - Update new codegen to work with latest Hoopl
  - Add some notes on new code gen to cmm-notes
  - Enable Hoopl lag package.
  - Add SPJ note to cmm-notes
  - Improve GC calls on new code generator.

Work in this branch was done by:
   - Milan Straka <fox@ucw.cz>
   - John Dias <dias@cs.tufts.edu>
   - David Terei <davidterei@gmail.com>

Edward Z. Yang <ezyang@mit.edu> merged in further changes from GHC HEAD
and fixed a few bugs.

It adds a third case to StgOp which already hold StgPrimOp and StgFCallOp.
The code generation for the new StgPrimCallOp case is almost exactly the
same as for out-of-line primops. They now share the tailCallPrim function.
In the Core -> STG translation we map foreign calls using the "prim"
calling convention to the StgPrimCallOp case. This is because in Core we
represent prim calls using the ForeignCall stuff. At the STG level however
the prim calls are really much more like primops than foreign calls.

- yet another wrong calling convention; this one was a special case for returning one
  value.

Since we introduced pointer tagging, we no longer always enter a
closure to evaluate it.  However, the biographical profiler relies on
closures being entered in order to mark them as "used", so we were
getting spurious amounts of data attributed to VOID.  It turns out
there are various places that need to be fixed, and I think at least
one of them was also wrong before pointer tagging (CgCon.cgReturnDataCon).

We used to use StaticFlags

This merge does not turn on the new codegen (which only compiles
a select few programs at this point),
but it does introduce some changes to the old code generator.

The high bits:
1. The Rep Swamp patch is finally here.
   The highlight is that the representation of types at the
   machine level has changed.
   Consequently, this patch contains updates across several back ends.
2. The new Stg -> Cmm path is here, although it appears to have a
   fair number of bugs lurking.
3. Many improvements along the CmmCPSZ path, including:
   o stack layout
   o some code for infotables, half of which is right and half wrong
   o proc-point splitting

Modules that need it import it themselves instead.

Changes too numerous to comment on, but here is some old history that
I saved: 


Wed Aug 15 11:07:13 BST 2007  Norman Ramsey <nr@eecs.harvard.edu>
  * type synonyms made consistent with new Cmm types

    M ./compiler/nativeGen/MachInstrs.hs -2 +2

Mon Aug 20 19:22:14 BST 2007  Norman Ramsey <nr@eecs.harvard.edu>
  * pushing return info beyond cmm into codegen

    M ./compiler/codeGen/Bitmap.hs r3
    M ./compiler/codeGen/CgBindery.lhs r3
    M ./compiler/codeGen/CgCallConv.hs r3
    M ./compiler/codeGen/CgCase.lhs r3
    M ./compiler/codeGen/CgClosure.lhs r3
    M ./compiler/codeGen/CgCon.lhs r3
    M ./compiler/codeGen/CgExpr.lhs r3
    M ./compiler/codeGen/CgForeignCall.hs -6 +7 r3
    M ./compiler/codeGen/CgHeapery.lhs r3
    M ./compiler/codeGen/CgHpc.hs +1 r3
    M ./compiler/codeGen/CgInfoTbls.hs r3
    M ./compiler/codeGen/CgLetNoEscape.lhs r3
    M ./compiler/codeGen/CgMonad.lhs r3
    M ./compiler/codeGen/CgParallel.hs r3
    M ./compiler/codeGen/CgPrimOp.hs +3 r3
    M ./compiler/codeGen/CgProf.hs r3
    M ./compiler/codeGen/CgStackery.lhs r3
    M ./compiler/codeGen/CgTailCall.lhs r3
    M ./compiler/codeGen/CgTicky.hs r3
    M ./compiler/codeGen/CgUtils.hs -1 +1 r3
    M ./compiler/codeGen/ClosureInfo.lhs r3
    M ./compiler/codeGen/CodeGen.lhs r3
    M ./compiler/codeGen/SMRep.lhs r3
    M ./compiler/nativeGen/AsmCodeGen.lhs -2 +2 r1
    M ./compiler/nativeGen/MachCodeGen.hs -3 +3 r1
    M ./compiler/nativeGen/MachInstrs.hs r1
    M ./compiler/nativeGen/MachRegs.lhs r1
    M ./compiler/nativeGen/NCGMonad.hs r1
    M ./compiler/nativeGen/PositionIndependentCode.hs r1
    M ./compiler/nativeGen/PprMach.hs r1
    M ./compiler/nativeGen/RegAllocInfo.hs r1
    M ./compiler/nativeGen/RegisterAlloc.hs r1

Mon Aug 20 20:54:41 BST 2007  Norman Ramsey <nr@eecs.harvard.edu>
  * put CmmReturnInfo into a CmmCall (and related types)

    M ./compiler/cmm/Cmm.hs -2 +1 r3
    M ./compiler/cmm/CmmBrokenBlock.hs -13 +12 r1
    M ./compiler/cmm/CmmCPS.hs -3 +3
    M ./compiler/cmm/CmmCPSGen.hs -8 +6 r1
    M ./compiler/cmm/CmmLint.hs -1 +1
    M ./compiler/cmm/CmmLive.hs -1 +1
    M ./compiler/cmm/CmmOpt.hs -3 +3
    M ./compiler/cmm/CmmParse.y -6 +6 r3
    M ./compiler/cmm/PprC.hs -3 +3
    M ./compiler/cmm/PprCmm.hs -7 +4 r2
    M ./compiler/codeGen/CgForeignCall.hs -7 +6 r2
    M ./compiler/codeGen/CgHpc.hs -1 r1
    M ./compiler/codeGen/CgPrimOp.hs -3 r1
    M ./compiler/codeGen/CgUtils.hs -1 +1 r1
    M ./compiler/nativeGen/AsmCodeGen.lhs -2 +2
    M ./compiler/nativeGen/MachCodeGen.hs -3 +3 r1

Tue Aug 21 18:09:13 BST 2007  Norman Ramsey <nr@eecs.harvard.edu>
  * add call info in nativeGen

    M ./compiler/nativeGen/AsmCodeGen.lhs r1
    M ./compiler/nativeGen/MachInstrs.hs r1
    M ./compiler/nativeGen/MachRegs.lhs r1
    M ./compiler/nativeGen/NCGMonad.hs r1
    M ./compiler/nativeGen/PositionIndependentCode.hs r1
    M ./compiler/nativeGen/PprMach.hs r1
    M ./compiler/nativeGen/RegAllocInfo.hs r1

Wed Aug 22 16:41:58 BST 2007  Norman Ramsey <nr@eecs.harvard.edu>
  * ListGraph is now a newtype, not a synonym
  The resultant bookkeepping is unenviable, but the change
  greatly simplifies our ability to make Cmm things propertly
  Outputable for both list-graph and zipper-graph representations.

    M ./compiler/cmm/Cmm.hs -5 +3
    M ./compiler/cmm/CmmCPS.hs -2 +2
    M ./compiler/cmm/CmmCPSGen.hs -1 +1
    M ./compiler/cmm/CmmContFlowOpt.hs -3 +3
    M ./compiler/cmm/CmmCvt.hs -2 +2
    M ./compiler/cmm/CmmInfo.hs -2 +3
    M ./compiler/cmm/CmmLint.hs -1 +1
    M ./compiler/cmm/CmmOpt.hs -2 +2
    M ./compiler/cmm/PprC.hs -1 +1
    M ./compiler/cmm/PprCmm.hs -5 +8
    M ./compiler/cmm/PprCmmZ.hs -7 +1
    M ./compiler/codeGen/CgMonad.lhs -1 +1
    M ./compiler/nativeGen/AsmCodeGen.lhs -15 +15
    M ./compiler/nativeGen/MachCodeGen.hs -2 +2
    M ./compiler/nativeGen/PositionIndependentCode.hs -6 +6
    M ./compiler/nativeGen/PprMach.hs -3 +2
    M ./compiler/nativeGen/RegAllocColor.hs +1
    M ./compiler/nativeGen/RegAllocLinear.hs -4 +5
    M ./compiler/nativeGen/RegCoalesce.hs -6 +6
    M ./compiler/nativeGen/RegLiveness.hs -12 +12

Thu Aug 23 13:44:49 BST 2007  Norman Ramsey <nr@eecs.harvard.edu>
  * diagnostic assistance in case fromJust fails

    M ./compiler/nativeGen/MachCodeGen.hs -2 +5

Thu Aug 23 14:07:28 BST 2007  Norman Ramsey <nr@eecs.harvard.edu>
  * give every block, even the first, a label
    With branch-chain elimination, the first block of a procedure
    might be the target of a branch.  This actually happens to 
    a dozen or more procedures in the run-time system.

    M ./compiler/nativeGen/PprMach.hs -8 +3

Fri Aug 24 17:27:04 BST 2007  Norman Ramsey <nr@eecs.harvard.edu>
  * clean up the code in PprMach

    M ./compiler/nativeGen/PprMach.hs -16 +14

Fri Aug 24 19:35:03 BST 2007  Norman Ramsey <nr@eecs.harvard.edu>
  * a bunch of impedance matching to get the compiler to build, plus 
   * the plus is diagnostics for unreachable code, which required
     moving a lot of prettyprinting code

    M ./compiler/cmm/Cmm.hs -7 +5
    M ./compiler/cmm/CmmCPSZ.hs -1 +1
    M ./compiler/cmm/CmmCvt.hs -8 +8
    M ./compiler/cmm/CmmParse.y -4 +3
    M ./compiler/cmm/MkZipCfg.hs -19 +9
    M ./compiler/cmm/PprCmmZ.hs -118 +4
    M ./compiler/cmm/ZipCfg.hs -1 +13
    M ./compiler/cmm/ZipCfgCmm.hs -10 +129
    M ./compiler/main/HscMain.lhs -4 +4
    M ./compiler/nativeGen/NCGMonad.hs -2 +2
    M ./compiler/nativeGen/RegAllocInfo.hs -3 +3

Fri Aug 31 14:38:02 BST 2007  Norman Ramsey <nr@eecs.harvard.edu>
  * fix a warning about an import

    M ./compiler/nativeGen/RegAllocColor.hs -1 +1

Older GHCs can't parse OPTIONS_GHC.
This also changes the URL referenced for the -w options from
WorkingConventions#Warnings to CodingStyle#Warnings for the compiler
modules.

This was causing us to try to jump to the address of an infotable when
unregisterised, leading to a segfault.

Instead of attaching the information whether a Label is going to be
accessed dynamically or not (distinction between IdLabel/DynLabel and
additional flags in ModuleInitLabel and PlainModuleInitLabel), we hand
dflags through the CmmOpt monad and the NatM monad. Before calling
labelDynamic in PositionIndependentCode, we extract thisPackage from
dflags and supply the current package to labelDynamic, so it can take
this information into account instead of extracting it from the labels
itself. This simplifies a lot of code in codeGen that just hands
through this_pkg.

  
This patch implements pointer tagging as per our ICFP'07 paper "Faster
laziness using dynamic pointer tagging".  It improves performance by
10-15% for most workloads, including GHC itself.

The original patches were by Alexey Rodriguez Yakushev
<mrchebas@gmail.com>, with additions and improvements by me.  I've
re-recorded the development as a single patch.

The basic idea is this: we use the low 2 bits of a pointer to a heap
object (3 bits on a 64-bit architecture) to encode some information
about the object pointed to.  For a constructor, we encode the "tag"
of the constructor (e.g. True vs. False), for a function closure its
arity.  This enables some decisions to be made without dereferencing
the pointer, which speeds up some common operations.  In particular it
enables us to avoid costly indirect jumps in many cases.

More information in the commentary:

http://hackage.haskell.org/trac/ghc/wiki/Commentary/Rts/HaskellExecution/PointerTagging

We recently discovered that they aren't a win any more, and just cost
code size.

In the generated code for case-of-variable, test the tag of the
scrutinee closure and only enter if it is unevaluated.  Also turn
*off* vectored returns.

In the generated code for case-of-variable, test the tag of the
scrutinee closure and only enter if it is unevaluated.  Also turn
*off* vectored returns.

This patch is a start on removing import lists and generally tidying
up the top of each module.  In addition to removing import lists:

   - Change DATA.IOREF -> Data.IORef etc.
   - Change List -> Data.List etc.
   - Remove $Id$
   - Update copyrights
   - Re-order imports to put non-GHC imports last
   - Remove some unused and duplicate imports

This patch pushes through one fundamental change: a module is now
identified by the pair of its package and module name, whereas
previously it was identified by its module name alone.  This means
that now a program can contain multiple modules with the same name, as
long as they belong to different packages.

This is a language change - the Haskell report says nothing about
packages, but it is now necessary to understand packages in order to
understand GHC's module system.  For example, a type T from module M
in package P is different from a type T from module M in package Q.
Previously this wasn't an issue because there could only be a single
module M in the program.

The "module restriction" on combining packages has therefore been
lifted, and a program can contain multiple versions of the same
package.

Note that none of the proposed syntax changes have yet been
implemented, but the architecture is geared towards supporting import
declarations qualified by package name, and that is probably the next
step.

It is now necessary to specify the package name when compiling a
package, using the -package-name flag (which has been un-deprecated).
Fortunately Cabal still uses -package-name.

Certain packages are "wired in".  Currently the wired-in packages are:
base, haskell98, template-haskell and rts, and are always referred to
by these versionless names.  Other packages are referred to with full
package IDs (eg. "network-1.0").  This is because the compiler needs
to refer to entities in the wired-in packages, and we didn't want to
bake the version of these packages into the comiler.  It's conceivable
that someone might want to upgrade the base package independently of
GHC.

Internal changes:

  - There are two module-related types:

        ModuleName      just a FastString, the name of a module
        Module          a pair of a PackageId and ModuleName

    A mapping from ModuleName can be a UniqFM, but a mapping from Module
    must be a FiniteMap (we provide it as ModuleEnv).

  - The "HomeModules" type that was passed around the compiler is now
    gone, replaced in most cases by the current package name which is
    contained in DynFlags.  We can tell whether a Module comes from the
    current package by comparing its package name against the current
    package.

  - While I was here, I changed PrintUnqual to be a little more useful:
    it now returns the ModuleName that the identifier should be qualified
    with according to the current scope, rather than its original
    module.  Also, PrintUnqual tells whether to qualify module names with
    package names (currently unused).

Docs to follow.

Most of the other users of the fptools build system have migrated to
Cabal, and with the move to darcs we can now flatten the source tree
without losing history, so here goes.

The main change is that the ghc/ subdir is gone, and most of what it
contained is now at the top level.  The build system now makes no
pretense at being multi-project, it is just the GHC build system.

No doubt this will break many things, and there will be a period of
instability while we fix the dependencies.  A straightforward build
should work, but I haven't yet fixed binary/source distributions.
Changes to the Building Guide will follow, too.

We now have more stg_ap entry points: stg_ap_*_fast, which take
arguments in registers according to the platform calling convention.
This is faster if the function being called is evaluated and has the
right arity, which is the common case (see the eval/apply paper for
measurements).  

We still need the stg_ap_*_info entry points for stack-based
application, such as an overflows when a function is applied to too
many argumnets.  The stg_ap_*_fast functions actually just check for
an evaluated function, and if they don't find one, push the args on
the stack and invoke stg_ap_*_info.  (this might be slightly slower in
some cases, but not the common case).

Relax the restrictions on conflicting packages.  This should address
many of the traps that people have been falling into with the current
package story.

Now, a local module can shadow a module in an exposed package, as long
as the package is not otherwise required by the program.  GHC checks
for conflicts when it knows the dependencies of the module being
compiled.

Also, we now check for module conflicts in exposed packages only when
importing a module: if an import can be satisfied from multiple
packages, that's an error.  It's not possible to prevent GHC from
starting by installing packages now (unless you install another base
package).

It seems to be possible to confuse GHCi by having a local module
shadowing a package module that goes away and comes back again.  I
think it's nearly right, but strange happenings have been observed.

I'll try to merge this into the STABLE branch.

Tweaks to get the GHC sources through Haddock.  Doesn't quite work
yet, because Haddock complains about the recursive modules.  Haddock
needs to understand SOURCE imports (it can probably just ignore them
as a first attempt).

Further integration with the new package story.  GHC now supports
pretty much everything in the package proposal.

  - GHC now works in terms of PackageIds (<pkg>-<version>) rather than
    just package names.  You can still specify package names without
    versions on the command line, as long as the name is unambiguous.

  - GHC understands hidden/exposed modules in a package, and will refuse
    to import a hidden module.  Also, the hidden/eposed status of packages
    is taken into account.

  - I had to remove the old package syntax from ghc-pkg, backwards
    compatibility isn't really practical.

  - All the package.conf.in files have been rewritten in the new syntax,
    and contain a complete list of modules in the package.  I've set all
    the versions to 1.0 for now - please check your package(s) and fix the
    version number & other info appropriately.

  - New options:

	-hide-package P    sets the expose flag on package P to False
	-ignore-package P  unregisters P for this compilation

	For comparison, -package P sets the expose flag on package P
        to True, and also causes P to be linked in eagerly.

        -package-name is no longer officially supported.  Unofficially, it's
	a synonym for -ignore-package, which has more or less the same effect
	as -package-name used to.

	Note that a package may be hidden and yet still be linked into
	the program, by virtue of being a dependency of some other package.
	To completely remove a package from the compiler's internal database,
        use -ignore-package.

	The compiler will complain if any two packages in the
        transitive closure of exposed packages contain the same
        module.

	You *must* use -ignore-package P when compiling modules for
        package P, if package P (or an older version of P) is already
        registered.  The compiler will helpfully complain if you don't.
	The fptools build system does this.

   - Note: the Cabal library won't work yet.  It still thinks GHC uses
     the old package config syntax.

Internal changes/cleanups:

   - The ModuleName type has gone away.  Modules are now just (a
     newtype of) FastStrings, and don't contain any package information.
     All the package-related knowledge is in DynFlags, which is passed
     down to where it is needed.

   - DynFlags manipulation has been cleaned up somewhat: there are no
     global variables holding DynFlags any more, instead the DynFlags
     are passed around properly.

   - There are a few less global variables in GHC.  Lots more are
     scheduled for removal.

   - -i is now a dynamic flag, as are all the package-related flags (but
     using them in {-# OPTIONS #-} is Officially Not Recommended).

   - make -j now appears to work under fptools/libraries/.  Probably
     wouldn't take much to get it working for a whole build.

------------------------------------
	Add Generalised Algebraic Data Types
	------------------------------------

This rather big commit adds support for GADTs.  For example,

    data Term a where
 	  Lit :: Int -> Term Int
	  App :: Term (a->b) -> Term a -> Term b
	  If  :: Term Bool -> Term a -> Term a
	  ..etc..

    eval :: Term a -> a
    eval (Lit i) = i
    eval (App a b) = eval a (eval b)
    eval (If p q r) | eval p    = eval q
    		    | otherwise = eval r


Lots and lots of of related changes throughout the compiler to make
this fit nicely.

One important change, only loosely related to GADTs, is that skolem
constants in the typechecker are genuinely immutable and constant, so
we often get better error messages from the type checker.  See
TcType.TcTyVarDetails.

There's a new module types/Unify.lhs, which has purely-functional
unification and matching for Type. This is used both in the typechecker
(for type refinement of GADTs) and in Core Lint (also for type refinement).

Merge backend-hacking-branch onto HEAD.  Yay!

Prune imports

Change the way SRTs are represented:

Previously, the SRT associated with a function or thunk would be a
sub-list of the enclosing top-level function's SRT.  But this approach
can lead to lots of duplication: if a CAF is referenced in several
different thunks, then it may appear several times in the SRT.
Let-no-escapes compound the problem, because the occurrence of a
let-no-escape-bound variable would expand to all the CAFs referred to
by the let-no-escape.

The new way is to describe the SRT associated with a function or thunk
as a (pointer+offset,bitmap) pair, where the pointer+offset points
into some SRT table (the enclosing function's SRT), and the bitmap
indicates which entries in this table are "live" for this closure.
The bitmap is stored in the 16 bits previously used for the length
field, but this rarely overflows.  When it does overflow, we store the
bitmap externally in a new "SRT descriptor".

Now the enclosing SRT can be a set, hence eliminating the duplicates.

Also, we now have one SRT per top-level function in a recursive group,
where previously we used to have one SRT for the whole group.  This
helps keep the size of SRTs down.

Bottom line: very little difference most of the time.  GHC itself got
slightly smaller.  One bad case of a module in GHC which had a huge
SRT has gone away.

While I was in the area:

  - Several parts of the back-end require bitmaps.  Functions for
    creating bitmaps are now centralised in the Bitmap module.

  - We were trying to be independent of word-size in a couple of
    places in the back end, but we've now abandoned that strategy so I
    simplified things a bit.

Merge the eval-apply-branch on to the HEAD
------------------------------------------

This is a change to GHC's evaluation model in order to ultimately make
GHC more portable and to reduce complexity in some areas.

At some point we'll update the commentary to describe the new state of
the RTS.  Pending that, the highlights of this change are:

  - No more Su.  The Su register is gone, update frames are one
    word smaller.

  - Slow-entry points and arg checks are gone.  Unknown function calls
    are handled by automatically-generated RTS entry points (AutoApply.hc,
    generated by the program in utils/genapply).

  - The stack layout is stricter: there are no "pending arguments" on
    the stack any more, the stack is always strictly a sequence of
    stack frames.

    This means that there's no need for LOOKS_LIKE_GHC_INFO() or
    LOOKS_LIKE_STATIC_CLOSURE() any more, and GHC doesn't need to know
    how to find the boundary between the text and data segments (BIG WIN!).

  - A couple of nasty hacks in the mangler caused by the neet to
    identify closure ptrs vs. info tables have gone away.

  - Info tables are a bit more complicated.  See InfoTables.h for the
    details.

  - As a side effect, GHCi can now deal with polymorphic seq.  Some bugs
    in GHCi which affected primitives and unboxed tuples are now
    fixed.

  - Binary sizes are reduced by about 7% on x86.  Performance is roughly
    similar, some programs get faster while some get slower.  I've seen
    GHCi perform worse on some examples, but haven't investigated
    further yet (GHCi performance *should* be about the same or better
    in theory).

  - Internally the code generator is rather better organised.  I've moved
    info-table generation from the NCG into the main codeGen where it is
    shared with the C back-end; info tables are now emitted as arrays
    of words in both back-ends.  The NCG is one step closer to being able
    to support profiling.

This has all been fairly thoroughly tested, but no doubt I've messed
up the commit in some way.