GitLab

Also:
 - improvements to code generation: push slow-call continuations
   on the stack instead of generating explicit continuations

 - remove unused CmmInfo wrapper type (replace with CmmInfoTable)

 - squash Area and AreaId together, remove now-unused RegSlot

 - comment out old unused stack-allocation code that no longer
   compiles after removal of RegSlot

We only use it for "compiler" sources, i.e. not for libraries.
Many modules have a -fno-warn-tabs kludge for now.

   CmmTop -> CmmDecl
   CmmPgm -> CmmGroup

I observed that the [CmmStatics] within CmmData uses the list in a very stylised way.
The first item in the list is almost invariably a CmmDataLabel. Many parts of the
compiler pattern match on this list and fail if this is not true.

This patch makes the invariant explicit by introducing a structured type CmmStatics
that holds the label and the list of remaining [CmmStatic].

There is one wrinkle: the x86 backend sometimes wants to output an alignment directive just
before the label. However, this can be easily fixed up by parameterising the native codegen
over the type of CmmStatics (though the GenCmmTop parameterisation) and using a pair
(Alignment, CmmStatics) there instead.

As a result, I think we will be able to remove CmmAlign and CmmDataLabel from the CmmStatic
data type, thus nuking a lot of code and failing pattern matches. This change will come as part
of my next patch.

Signed-off-by: Edward Z. Yang <ezyang@mit.edu>

The following patches were ported:

    d0faaa6f Fix segfault in array copy primops on 32-bit
    18691d44 Make assignTemp_ less pessimistic
    9c23f06f Make array copy primops inline
Signed-off-by: Edward Z. Yang <ezyang@mit.edu>

This changes the new code generator to make use of the Hoopl package
for dataflow analysis.  Hoopl is a new boot package, and is maintained
in a separate upstream git repository (as usual, GHC has its own
lagging darcs mirror in http://darcs.haskell.org/packages/hoopl).

During this merge I squashed recent history into one patch.  I tried
to rebase, but the history had some internal conflicts of its own
which made rebase extremely confusing, so I gave up. The history I
squashed was:

  - Update new codegen to work with latest Hoopl
  - Add some notes on new code gen to cmm-notes
  - Enable Hoopl lag package.
  - Add SPJ note to cmm-notes
  - Improve GC calls on new code generator.

Work in this branch was done by:
   - Milan Straka <fox@ucw.cz>
   - John Dias <dias@cs.tufts.edu>
   - David Terei <davidterei@gmail.com>

Edward Z. Yang <ezyang@mit.edu> merged in further changes from GHC HEAD
and fixed a few bugs.

This replaces the global blackhole_queue with a clever scheme that
enables us to queue up blocked threads on the closure that they are
blocked on, while still avoiding atomic instructions in the common
case.

Advantages:

 - gets rid of a locked global data structure and some tricky GC code
   (replacing it with some per-thread data structures and different
   tricky GC code :)

 - wakeups are more prompt: parallel/concurrent performance should
   benefit.  I haven't seen anything dramatic in the parallel
   benchmarks so far, but a couple of threading benchmarks do improve
   a bit.

 - waking up a thread blocked on a blackhole is now O(1) (e.g. if
   it is the target of throwTo).

 - less sharing and better separation of Capabilities: communication
   is done with messages, the data structures are strictly owned by a
   Capability and cannot be modified except by sending messages.

 - this change will utlimately enable us to do more intelligent
   scheduling when threads block on each other.  This is what started
   off the whole thing, but it isn't done yet (#3838).

I'll be documenting all this on the wiki in due course.

The type of the CmmLabel ctor is now
  CmmLabel :: PackageId -> FastString -> CmmLabelInfo -> CLabel
  
 - When you construct a CmmLabel you have to explicitly say what
   package it is in. Many of these will just use rtsPackageId, but
   I've left it this way to remind people not to pretend labels are
   in the RTS package when they're not. 
   
 - When parsing a Cmm file, labels that are not defined in the 
   current file are assumed to be in the RTS package. 
   
   Labels imported like
      import label
   are assumed to be in a generic "foreign" package, which is different
   from the current one.
   
   Labels imported like
      import "package-name" label
   are marked as coming from the named package.
   
   This last one is needed for the integer-gmp library as we want to
   refer to labels that are not in the same compilation unit, but
   are in the same non-rts package.
   
   This should help remove the nasty #ifdef __PIC__ stuff from
   integer-gmp/cbits/gmp-wrappers.cmm
   

This merge does not turn on the new codegen (which only compiles
a select few programs at this point),
but it does introduce some changes to the old code generator.

The high bits:
1. The Rep Swamp patch is finally here.
   The highlight is that the representation of types at the
   machine level has changed.
   Consequently, this patch contains updates across several back ends.
2. The new Stg -> Cmm path is here, although it appears to have a
   fair number of bugs lurking.
3. Many improvements along the CmmCPSZ path, including:
   o stack layout
   o some code for infotables, half of which is right and half wrong
   o proc-point splitting

o Moved BlockId stuff to a new file to avoid module recursion
o Defined stack areas for parameter-passing locations and spill slots
o Part way through replacing copy in and copy out nodes
  - added movement instructions for stack pointer
  - added movement instructions for call and return parameters
    (but not with the proper calling conventions)
o Inserting spills and reloads for proc points is now procpoint-aware
  (it was relying on the presence of a CopyIn node as a proxy for
   procpoint knowledge)
o Changed ZipDataflow to expect AGraphs (instead of being polymorphic in
   the type of graph)

I've renamed a number of type and data constructors within Cmm so that
the names used in the compiler may more closely reflect the C--
specification 2.1.  I've done a bit of other renaming as well.
Highlights:

  CmmFormal and CmmActual now bear a CmmKind (which for now is a
                                              MachHint as before)
  CmmFormals = [CmmFormal] and CmmActuals = [CmmActual]
  
  suitable changes have been made to both code and nonterminals in the
  Cmm parser (which is as yet untested)

  For reasons I don't understand, parts of the code generator use a
  sequence of 'formal parameters' with no C-- kinds.  For these we now
  have the types
    type CmmFormalWithoutKind   = LocalReg
    type CmmFormalsWithoutKinds = [CmmFormalWithoutKind]

  A great many appearances of (Tau, MachHint) have been simplified to
  the appropriate CmmFormal or CmmActual, though I'm sure there are
  more opportunities.

  Kind and its data constructors are now renamed to
     data GCKind = GCKindPtr | GCKindNonPtr 
  to avoid confusion with the Kind used in the type checker and with CmmKind.

Finally, in a somewhat unrelated bit (and in honor of Simon PJ, who
thought of the name), the Whalley/Davidson 'transaction limit' is now
called 'OptimizationFuel' with the net effect that there are no longer
two unrelated uses of the abbreviation 'tx'.

Changes too numerous to comment on, but here is some old history that
I saved: 


Wed Aug 15 11:07:13 BST 2007  Norman Ramsey <nr@eecs.harvard.edu>
  * type synonyms made consistent with new Cmm types

    M ./compiler/nativeGen/MachInstrs.hs -2 +2

Mon Aug 20 19:22:14 BST 2007  Norman Ramsey <nr@eecs.harvard.edu>
  * pushing return info beyond cmm into codegen

    M ./compiler/codeGen/Bitmap.hs r3
    M ./compiler/codeGen/CgBindery.lhs r3
    M ./compiler/codeGen/CgCallConv.hs r3
    M ./compiler/codeGen/CgCase.lhs r3
    M ./compiler/codeGen/CgClosure.lhs r3
    M ./compiler/codeGen/CgCon.lhs r3
    M ./compiler/codeGen/CgExpr.lhs r3
    M ./compiler/codeGen/CgForeignCall.hs -6 +7 r3
    M ./compiler/codeGen/CgHeapery.lhs r3
    M ./compiler/codeGen/CgHpc.hs +1 r3
    M ./compiler/codeGen/CgInfoTbls.hs r3
    M ./compiler/codeGen/CgLetNoEscape.lhs r3
    M ./compiler/codeGen/CgMonad.lhs r3
    M ./compiler/codeGen/CgParallel.hs r3
    M ./compiler/codeGen/CgPrimOp.hs +3 r3
    M ./compiler/codeGen/CgProf.hs r3
    M ./compiler/codeGen/CgStackery.lhs r3
    M ./compiler/codeGen/CgTailCall.lhs r3
    M ./compiler/codeGen/CgTicky.hs r3
    M ./compiler/codeGen/CgUtils.hs -1 +1 r3
    M ./compiler/codeGen/ClosureInfo.lhs r3
    M ./compiler/codeGen/CodeGen.lhs r3
    M ./compiler/codeGen/SMRep.lhs r3
    M ./compiler/nativeGen/AsmCodeGen.lhs -2 +2 r1
    M ./compiler/nativeGen/MachCodeGen.hs -3 +3 r1
    M ./compiler/nativeGen/MachInstrs.hs r1
    M ./compiler/nativeGen/MachRegs.lhs r1
    M ./compiler/nativeGen/NCGMonad.hs r1
    M ./compiler/nativeGen/PositionIndependentCode.hs r1
    M ./compiler/nativeGen/PprMach.hs r1
    M ./compiler/nativeGen/RegAllocInfo.hs r1
    M ./compiler/nativeGen/RegisterAlloc.hs r1

Mon Aug 20 20:54:41 BST 2007  Norman Ramsey <nr@eecs.harvard.edu>
  * put CmmReturnInfo into a CmmCall (and related types)

    M ./compiler/cmm/Cmm.hs -2 +1 r3
    M ./compiler/cmm/CmmBrokenBlock.hs -13 +12 r1
    M ./compiler/cmm/CmmCPS.hs -3 +3
    M ./compiler/cmm/CmmCPSGen.hs -8 +6 r1
    M ./compiler/cmm/CmmLint.hs -1 +1
    M ./compiler/cmm/CmmLive.hs -1 +1
    M ./compiler/cmm/CmmOpt.hs -3 +3
    M ./compiler/cmm/CmmParse.y -6 +6 r3
    M ./compiler/cmm/PprC.hs -3 +3
    M ./compiler/cmm/PprCmm.hs -7 +4 r2
    M ./compiler/codeGen/CgForeignCall.hs -7 +6 r2
    M ./compiler/codeGen/CgHpc.hs -1 r1
    M ./compiler/codeGen/CgPrimOp.hs -3 r1
    M ./compiler/codeGen/CgUtils.hs -1 +1 r1
    M ./compiler/nativeGen/AsmCodeGen.lhs -2 +2
    M ./compiler/nativeGen/MachCodeGen.hs -3 +3 r1

Tue Aug 21 18:09:13 BST 2007  Norman Ramsey <nr@eecs.harvard.edu>
  * add call info in nativeGen

    M ./compiler/nativeGen/AsmCodeGen.lhs r1
    M ./compiler/nativeGen/MachInstrs.hs r1
    M ./compiler/nativeGen/MachRegs.lhs r1
    M ./compiler/nativeGen/NCGMonad.hs r1
    M ./compiler/nativeGen/PositionIndependentCode.hs r1
    M ./compiler/nativeGen/PprMach.hs r1
    M ./compiler/nativeGen/RegAllocInfo.hs r1

Wed Aug 22 16:41:58 BST 2007  Norman Ramsey <nr@eecs.harvard.edu>
  * ListGraph is now a newtype, not a synonym
  The resultant bookkeepping is unenviable, but the change
  greatly simplifies our ability to make Cmm things propertly
  Outputable for both list-graph and zipper-graph representations.

    M ./compiler/cmm/Cmm.hs -5 +3
    M ./compiler/cmm/CmmCPS.hs -2 +2
    M ./compiler/cmm/CmmCPSGen.hs -1 +1
    M ./compiler/cmm/CmmContFlowOpt.hs -3 +3
    M ./compiler/cmm/CmmCvt.hs -2 +2
    M ./compiler/cmm/CmmInfo.hs -2 +3
    M ./compiler/cmm/CmmLint.hs -1 +1
    M ./compiler/cmm/CmmOpt.hs -2 +2
    M ./compiler/cmm/PprC.hs -1 +1
    M ./compiler/cmm/PprCmm.hs -5 +8
    M ./compiler/cmm/PprCmmZ.hs -7 +1
    M ./compiler/codeGen/CgMonad.lhs -1 +1
    M ./compiler/nativeGen/AsmCodeGen.lhs -15 +15
    M ./compiler/nativeGen/MachCodeGen.hs -2 +2
    M ./compiler/nativeGen/PositionIndependentCode.hs -6 +6
    M ./compiler/nativeGen/PprMach.hs -3 +2
    M ./compiler/nativeGen/RegAllocColor.hs +1
    M ./compiler/nativeGen/RegAllocLinear.hs -4 +5
    M ./compiler/nativeGen/RegCoalesce.hs -6 +6
    M ./compiler/nativeGen/RegLiveness.hs -12 +12

Thu Aug 23 13:44:49 BST 2007  Norman Ramsey <nr@eecs.harvard.edu>
  * diagnostic assistance in case fromJust fails

    M ./compiler/nativeGen/MachCodeGen.hs -2 +5

Thu Aug 23 14:07:28 BST 2007  Norman Ramsey <nr@eecs.harvard.edu>
  * give every block, even the first, a label
    With branch-chain elimination, the first block of a procedure
    might be the target of a branch.  This actually happens to 
    a dozen or more procedures in the run-time system.

    M ./compiler/nativeGen/PprMach.hs -8 +3

Fri Aug 24 17:27:04 BST 2007  Norman Ramsey <nr@eecs.harvard.edu>
  * clean up the code in PprMach

    M ./compiler/nativeGen/PprMach.hs -16 +14

Fri Aug 24 19:35:03 BST 2007  Norman Ramsey <nr@eecs.harvard.edu>
  * a bunch of impedance matching to get the compiler to build, plus 
   * the plus is diagnostics for unreachable code, which required
     moving a lot of prettyprinting code

    M ./compiler/cmm/Cmm.hs -7 +5
    M ./compiler/cmm/CmmCPSZ.hs -1 +1
    M ./compiler/cmm/CmmCvt.hs -8 +8
    M ./compiler/cmm/CmmParse.y -4 +3
    M ./compiler/cmm/MkZipCfg.hs -19 +9
    M ./compiler/cmm/PprCmmZ.hs -118 +4
    M ./compiler/cmm/ZipCfg.hs -1 +13
    M ./compiler/cmm/ZipCfgCmm.hs -10 +129
    M ./compiler/main/HscMain.lhs -4 +4
    M ./compiler/nativeGen/NCGMonad.hs -2 +2
    M ./compiler/nativeGen/RegAllocInfo.hs -3 +3

Fri Aug 31 14:38:02 BST 2007  Norman Ramsey <nr@eecs.harvard.edu>
  * fix a warning about an import

    M ./compiler/nativeGen/RegAllocColor.hs -1 +1

The type parameter to a C-- procedure now represents a control-flow
graph, not a single instruction.  The newtype ListGraph preserves the 
current representation while enabling other representations and a
sensible way of prettyprinting.  Except for a few changes in the
prettyprinter the new compiler binary should be bit-for-bit identical
to the old.

Older GHCs can't parse OPTIONS_GHC.
This also changes the URL referenced for the -w options from
WorkingConventions#Warnings to CodingStyle#Warnings for the compiler
modules.

mapAccumL and mapAccumR are in Data.List now.
mapAccumB is unused.

(This required a bit of refactoring of CmmInfo.)

This eliminates one of the panics introduced by 
the previous patch:
'First pass at implementing info tables for CPS'
  
The other panic introduced by that patch still remains.
It was due to the need to convert from a
  ContinuationInfo to a CmmInfo.
  (codeGen/CgInfoTbls.hs:emitClosureCodeAndInfoTable)
  (codeGen/CgInfoTbls.hs:emitReturnTarget)

This is a fairly complete implementation, however
two 'panic's have been placed in the critical path
where the implementation is still a bit lacking so
do not expect it to run quite yet.

One call to panic is because we still need to create
a GC block for procedures that don't have them yet.
(cmm/CmmCPS.hs:continuationToProc)

The other is due to the need to convert from a
ContinuationInfo to a CmmInfo.
(codeGen/CgInfoTbls.hs:emitClosureCodeAndInfoTable)
(codeGen/CgInfoTbls.hs:emitReturnTarget)

Since a CmmCall returns CmmFormals which may include
global registers (and indeed one place in the code
returns the results of a CmmCall into BaseReg) and
since CPS conversion will change those return slots
into formal arguments for the continuation of the call,
CmmProc has to have CmmFormals for the formal arguments.

Oddly, the old code never made use of procedure arguments
so this change only effects the types and not any of the code.
(Because [] is both of type [LocalReg] and CmmFormals.)

We recently discovered that they aren't a win any more, and just cost
code size.

This patch is a start on removing import lists and generally tidying
up the top of each module.  In addition to removing import lists:

   - Change DATA.IOREF -> Data.IORef etc.
   - Change List -> Data.List etc.
   - Remove $Id$
   - Update copyrights
   - Re-order imports to put non-GHC imports last
   - Remove some unused and duplicate imports

This patch pushes through one fundamental change: a module is now
identified by the pair of its package and module name, whereas
previously it was identified by its module name alone.  This means
that now a program can contain multiple modules with the same name, as
long as they belong to different packages.

This is a language change - the Haskell report says nothing about
packages, but it is now necessary to understand packages in order to
understand GHC's module system.  For example, a type T from module M
in package P is different from a type T from module M in package Q.
Previously this wasn't an issue because there could only be a single
module M in the program.

The "module restriction" on combining packages has therefore been
lifted, and a program can contain multiple versions of the same
package.

Note that none of the proposed syntax changes have yet been
implemented, but the architecture is geared towards supporting import
declarations qualified by package name, and that is probably the next
step.

It is now necessary to specify the package name when compiling a
package, using the -package-name flag (which has been un-deprecated).
Fortunately Cabal still uses -package-name.

Certain packages are "wired in".  Currently the wired-in packages are:
base, haskell98, template-haskell and rts, and are always referred to
by these versionless names.  Other packages are referred to with full
package IDs (eg. "network-1.0").  This is because the compiler needs
to refer to entities in the wired-in packages, and we didn't want to
bake the version of these packages into the comiler.  It's conceivable
that someone might want to upgrade the base package independently of
GHC.

Internal changes:

  - There are two module-related types:

        ModuleName      just a FastString, the name of a module
        Module          a pair of a PackageId and ModuleName

    A mapping from ModuleName can be a UniqFM, but a mapping from Module
    must be a FiniteMap (we provide it as ModuleEnv).

  - The "HomeModules" type that was passed around the compiler is now
    gone, replaced in most cases by the current package name which is
    contained in DynFlags.  We can tell whether a Module comes from the
    current package by comparing its package name against the current
    package.

  - While I was here, I changed PrintUnqual to be a little more useful:
    it now returns the ModuleName that the identifier should be qualified
    with according to the current scope, rather than its original
    module.  Also, PrintUnqual tells whether to qualify module names with
    package names (currently unused).

Docs to follow.

Most of the other users of the fptools build system have migrated to
Cabal, and with the move to darcs we can now flatten the source tree
without losing history, so here goes.

The main change is that the ghc/ subdir is gone, and most of what it
contained is now at the top level.  The build system now makes no
pretense at being multi-project, it is just the GHC build system.

No doubt this will break many things, and there will be a period of
instability while we fix the dependencies.  A straightforward build
should work, but I haven't yet fixed binary/source distributions.
Changes to the Building Guide will follow, too.

Relax the restrictions on conflicting packages.  This should address
many of the traps that people have been falling into with the current
package story.

Now, a local module can shadow a module in an exposed package, as long
as the package is not otherwise required by the program.  GHC checks
for conflicts when it knows the dependencies of the module being
compiled.

Also, we now check for module conflicts in exposed packages only when
importing a module: if an import can be satisfied from multiple
packages, that's an error.  It's not possible to prevent GHC from
starting by installing packages now (unless you install another base
package).

It seems to be possible to confuse GHCi by having a local module
shadowing a package module that goes away and comes back again.  I
think it's nearly right, but strange happenings have been observed.

I'll try to merge this into the STABLE branch.

Flags cleanup.

Basically the purpose of this commit is to move more of the compiler's
global state into DynFlags, which is moving in the direction we need
to go for the GHC API which can have multiple active sessions
supported by a single GHC instance.

Before:

$ grep 'global_var' */*hs | wc -l
     78

After:

$ grep 'global_var' */*hs | wc -l
     27

Well, it's an improvement.  Most of what's left won't really affect
our ability to host multiple sessions.

Lots of static flags have become dynamic flags (yay!).  Notably lots
of flags that we used to think of as "driver" flags, like -I and -L,
are now dynamic.  The most notable static flags left behind are the
"way" flags, eg. -prof.  It would be nice to fix this, but it isn't
urgent.

On the way, lots of cleanup has happened.  Everything related to
static and dynamic flags lives in StaticFlags and DynFlags
respectively, and they share a common command-line parser library in
CmdLineParser.  The flags related to modes (--makde, --interactive
etc.) are now private to the front end: in fact private to Main
itself, for now.