GitLab

Merge backend-hacking-branch onto HEAD.  Yay!

Oops. Should import Directory instead of System.Directory.

Changed outputForeignStubs to check whether stub files from a previous
compilation still exist (in the case where no new stubs exist). This is
necessary to compile External Core programs that require these stubs.

Remove duplicate #includes arising from foreign import decls

A round of space-leak fixing.

  - re-instate zapping of the PersistentCompilerState at various
    points during the compilation cycle in HscMain.  This affects
    one-shot compilation only, since in this mode the information
    collected in the PCS is not required after creating the final
    interface file.

  - Unravel the recursive dependency between MkIface and
    CoreTidy/CoreToStg.  Previously the CafInfo for each binding was
    calculated by CoreToStg, and fed back into the IdInfo of the Ids
    generated by CoreTidy (an earlier pass).  MkIface then took this
    IdInfo and the bindings from CoreTidy to generate the interface;
    but it couldn't do this until *after* CoreToStg, because the CafInfo
    hadn't been calculated yet.  The result was that the CoreTidy
    output lived until after CoreToStg, and at the same time as the
    CorePrep and STG syntax, which is wasted space, not to mention
    the complexity and general ugliness in HscMain.

    So now we calculate CafInfo directly in CoreTidy.  The downside is
    that we have to predict what CorePrep is going to do to the
    bindings so we can tell what will turn into a CAF later, but it's
    no worse than before (it turned out that we were doing this
    prediction before in CoreToStg anyhow).

  - The typechecker lazilly typechecks unfoldings.  It turns out that
    this is a good idea from a performance perspective, but it also
    means that it must hang on to all the information it needs to
    do the typechecking.  Previously this meant holding on to the
    whole of the typechecker's environment, which includes all sorts
    of stuff which isn't necessary to typecheck unfoldings.  By paring
    down the environment captured by the lazy unfoldings, we can
    save quite a bit of space in the phases after typechecking.

Eliminate brain-dead outputC pattern-match failure

Add the #includes from the rts package to the stub .c file.

Prior to rev. 1.42, we used to add all the #includes from all enabled
packages, together with any -#include options from the command-line.
But since this is auto-generated code and we know exactly which
#includes are required, this was overkill.

In rev. 1.42, all #includes except RtsAPI.h were removed from the stub
.c file.  This was incorrect, because the stub file refers to some
entities defined in other RTS header files (the StgStablePtr type, and
deRefStablePtr() for example).

This change adds the header files from the rts package to the stub .c
file, fixing some recent test breakages.

"Auto" packages.

The big change here is that it is no longer necessary to explicitly
say '-package X' on the command line if X is a package containing
hierarchical Haskell modules.  All packages marked "auto" contribute
to the import path, so their modules are always available.  At link
time, the compiler knows which packages are actually used by the
program, and it links in only those libraries needed.

There's one exception: one-shot linking.  If you link a program using

    ghc -o prog A.o B.o ...

then you need to explicitly add -package flags for each package
required (except base & haskell98) because the compiler has no
information about the package dependencies in this case.

Package configs have a new field: auto, which is either True or False.
Non-auto packages must be mentioned on the command-line as usual.
Non-auto packages are still required for:

  - non-hierarchical libraries (to avoid polluting the module namespace)

  - packages with no Haskell content

  - if you want more than one version of a package, or packages
    providing overlapping functionality where the user must decide
    which one to use.

Doc changes to follow...

--------------------------------------
	Make Template Haskell into the HEAD
	--------------------------------------

This massive commit transfers to the HEAD all the stuff that
Simon and Tim have been doing on Template Haskell.  The
meta-haskell-branch is no more!

WARNING: make sure that you

  * Update your links if you are using link trees.
    Some modules have been added, some have gone away.

  * Do 'make clean' in all library trees.
    The interface file format has changed, and you can
    get strange panics (sadly) if GHC tries to read old interface files:
    e.g.  ghc-5.05: panic! (the `impossible' happened, GHC version 5.05):
	  Binary.get(TyClDecl): ForeignType

  * You need to recompile the rts too; Linker.c has changed


However the libraries are almost unaltered; just a tiny change in
Base, and to the exports in Prelude.


NOTE: so far as TH itself is concerned, expression splices work
fine, but declaration splices are not complete.


		---------------
		The main change
		---------------

The main structural change: renaming and typechecking have to be
interleaved, because we can't rename stuff after a declaration splice
until after we've typechecked the stuff before (and the splice
itself).

* Combine the renamer and typecheker monads into one
	(TcRnMonad, TcRnTypes)
  These two replace TcMonad and RnMonad

* Give them a single 'driver' (TcRnDriver).  This driver
  replaces TcModule.lhs and Rename.lhs

* The haskell-src library package has a module
	Language/Haskell/THSyntax
  which defines the Haskell data type seen by the TH programmer.

* New modules:
	hsSyn/Convert.hs 	converts THSyntax -> HsSyn
	deSugar/DsMeta.hs 	converts HsSyn -> THSyntax

* New module typecheck/TcSplice type-checks Template Haskell splices.

		-------------
		Linking stuff
		-------------

* ByteCodeLink has been split into
	ByteCodeLink	(which links)
	ByteCodeAsm	(which assembles)

* New module ghci/ObjLink is the object-code linker.

* compMan/CmLink is removed entirely (was out of place)
  Ditto CmTypes (which was tiny)

* Linker.c initialises the linker when it is first used (no need to call
  initLinker any more).  Template Haskell makes it harder to know when
  and whether to initialise the linker.


	-------------------------------------
	Gathering the LIE in the type checker
	-------------------------------------

* Instead of explicitly gathering constraints in the LIE
	tcExpr :: RenamedExpr -> TcM (TypecheckedExpr, LIE)
  we now dump the constraints into a mutable varabiable carried
  by the monad, so we get
	tcExpr :: RenamedExpr -> TcM TypecheckedExpr

  Much less clutter in the code, and more efficient too.
  (Originally suggested by Mark Shields.)


		-----------------
		Remove "SysNames"
		-----------------

Because the renamer and the type checker were entirely separate,
we had to carry some rather tiresome implicit binders (or "SysNames")
along inside some of the HsDecl data structures.  They were both
tiresome and fragile.

Now that the typechecker and renamer are more intimately coupled,
we can eliminate SysNames (well, mostly... default methods still
carry something similar).

		-------------
		Clean up HsPat
		-------------

One big clean up is this: instead of having two HsPat types (InPat and
OutPat), they are now combined into one.  This is more consistent with
the way that HsExpr etc is handled; there are some 'Out' constructors
for the type checker output.

So:
	HsPat.InPat	--> HsPat.Pat
	HsPat.OutPat	--> HsPat.Pat
	No 'pat' type parameter in HsExpr, HsBinds, etc

	Constructor patterns are nicer now: they use
		HsPat.HsConDetails
	for the three cases of constructor patterns:
		prefix, infix, and record-bindings

	The *same* data type HsConDetails is used in the type
	declaration of the data type (HsDecls.TyData)

Lots of associated clean-up operations here and there.  Less code.
Everything is wonderful.

Housekeeping:

  - The main goal is to remove dependencies on hslibs for a
    bootstrapped compiler, leaving only a requirement that the
    packages base, haskell98 and readline are built in stage 1 in
    order to bootstrap.  We're almost there: Posix is still required
    for signal handling, but all other dependencies on hslibs are now
    gone.

    Uses of Addr and ByteArray/MutableByteArray array are all gone
    from the compiler.  PrimPacked defines the Ptr type for GHC 4.08
    (which didn't have it), and it defines simple BA and MBA types to
    replace uses of ByteArray and MutableByteArray respectively.

  - Clean up import lists.  HsVersions.h now defines macros for some
    modules which have moved between GHC versions.  eg. one now
    imports 'GLAEXTS' to get at unboxed types and primops in the
    compiler.

    Many import lists have been sorted as per the recommendations in
    the new style guidelines in the commentary.

I've built the compiler with GHC 4.08.2, 5.00.2, 5.02.3, 5.04 and
itself, and everything still works here.  Doubtless I've got something
wrong, though.

An I/O error while opening/writing the output file is *not* a panic.

Make foreign export dynamic work in GHCi.  Main changes:

* Allow literal labels to propagate through the bytecode generator
  and eventually be linked by the runtime linker.

* Minor mods to driver plumbing so that GHCi produces the relevant
  *_stub.[ch] files, compiles them with gcc, and loads the resulting .o's

* Dereference the stable pointer in the generated C stub, rather
  than passing it to a Haskell-world helper.  This seems simpler and
  removes the need to have a H-world helper, which in turn means the
  stub .o doesn't refer to any H-world entities.  This is important
  because our linker can't deal with mutual recursion between
  BCOs and loaded objects.

Still ToDo:

* Make it thread/GC safe.  (Sigbjorn?)

* Get rid of the bits of code in DsForeign which generate the
  Haskell helper.  I had a go but it wasn't obvious how to do it,
  so have deferred.

Updates to the native code generator following the changes to fix the
large block allocation bug, and changes to use the new
function-address cache in the register table to reduce code size.

Also: I changed the pretty-printing machinery for assembly code to use
Pretty rather than Outputable, since we don't make use of the styles
and it should improve performance.  Perhaps the same should be done
for abstract C.

Make compilation of javaGen conditional on $(GhcWithJavaGen) in the
same way as the ILX and native backends.

Add C++ extern "C" wrappers to generated stubs.

Append final newline to _stub.{c,h} output

Various changes for ILX backend and type-passing compilers, code reviewed by SimonPJ

better "#define IN_STG_CODE 0" before including the standard HC header
in a foreign export dynamic stub, because this isn't really HC code.

Simplify the foreign-export stub processing.

  - DynFlags now has fields for the stub.h and stub.c filenames, for
    consistency with the normal hsc output file name.

  - codeOutput puts the stubs into these files rather than dreaming
    up new temporary names for them

  - now we don't have to move the stubs into the right place in
    DriverPipeline.

  - we do however have to inject the correct #includes into the stub.c
    file when it is generated: I'm now injecting the same includes as
    the .hc file gets plus "RtsAPI.h", which is probably more correct
    than the hacky hardcoded "Stg.h" we had before.

don't forget to inject a #include for the stub.h file.

Changes to support bootstrapping the compiler from .hc files.  It's
not quite working yet, but it's not far off.

  - the biggest change is that any injected #includes are now placed in
    the .hc file at generation time, rather than compilation time.  I
    can't see any reason not to do this - it makes it clear by looking at
    the .hc file which files are being #included, it means one less
    temporary file at compilation time, and it means the .hc file is more
    standalone.

  - all the gruesomeness is in mk/bootstrap.mk, which handles building
    .hc files without a ghc driver.

Remove debug print

----------------
	Nuke ClassContext
	----------------

This commit tidies up a long-standing inconsistency in GHC.
The context of a class or instance decl used to be restricted
to predicates of the form
	C t1 .. tn
with
	type ClassContext = [(Class,[Type])]

but everywhere else in the compiler we used

	type ThetaType = [PredType]
where PredType can be any sort of constraint (= predicate).

The inconsistency actually led to a crash, when compiling
	class (?x::Int) => C a where {}

I've tidied all this up by nuking ClassContext altogether, and using
PredType throughout.  Lots of modified files, but all in
more-or-less trivial ways.

I've also added a check that the context of a class or instance
decl doesn't include a non-inheritable predicate like (?x::Int).

Other things

 * rename constructor 'Class' from type TypeRep.Pred to 'ClassP'
   (makes it easier to grep for)

 * rename constructor HsPClass  => HsClassP
		      HsPIParam => HsIParam

Fix overenthusiatic import GCing, I presume.

----------------
	First cut at ILX
	----------------

This commit puts the ILX .NET code generator into the head.
It's entirely untested, mind you.

Some changes to the Module/Package strutures, mainly of a
naming variety.  In particular:

	Package	===> PackageConfig

Add ILX support (all #ifdefed on ILX for now) and tidied up some
indentation. We now support --mk-dll, so remove the comment about adding
that.

_scc_'s for the NCG

Unused imports and suchlike

Return the correct file name for .stub_[ch] files.

1.	Outputable.PprStyle now carries a bit more information
	In particular, the printing style tells whether to print
	a name in unqualified form.  This used to be embedded in
	a Name, but since Names now outlive a single compilation unit,
	that's no longer appropriate.

	So now the print-unqualified predicate is passed in the printing
	style, not embedded in the Name.

   2.	I tidied up HscMain a little.  Many of the showPass messages
	have migraged into the repective pass drivers

This commit completes the merge of compiler part
	of the HEAD with the before-ghci-branch to
        before-ghci-branch-merged.

More small changes

Finally get CompManager to compile.  Also rm some redundant imports.

* Stop pipeline when recompilation not needed.
* Check OPTIONS pragmas for non-dynamic flags.
* Misc wibbles.

Improve MkIface; get ready for NameEnv.lhs

Compile up to HscMain.  Again :)

Compile everything needed by main/HscMain.

Changes to make CodeOutput compile.

remove unused imports

Major thing: new register allocator.  Brief description follows.
Should correctly handle code with loops in, even though we don't
generate any such at the moment.  A lot of comments.  The previous
machinery for spilling is retained, as is the idea of a fast-and-easy
initial allocation attempt intended to deal with the majority of code
blocks (about 60% on x86) very cheaply.  Many comments explaining
in detail how it works :-)

The Stix inliner is now on by default.  Integer code seems to run
within about 1% of that -fvia-C.  x86 fp code is significantly worse,
up to about 30% slower, depending on the amount of fp activity.

Minor thing: lazyfication of the top-level NCG plumbing, so that the
NCG doesn't require any greater residency than compiling to C, just a
bit more time.  Created lazyThenUs and lazyMapUs for this purpose.

The new allocator is somewhat, although not catastophically, slower
than the old one.  Fixing of the long-standing NCG space leak more
than makes up for it; overall hsc run-time is down about 5%, due to
significantly reduced GC time.

--------------------------------------------------------------------

Instructions are numbered sequentially, starting at zero.

A flow edge (FE) is a pair of insn numbers (MkFE Int Int) denoting
a possible flow of control from the first insn to the second.

The input to the register allocator is a list of instructions, which
mention Regs.  A Reg can be a RealReg -- a real machine reg -- or a
VirtualReg, which carries a unique.  After allocation, all the
VirtualReg references will have been converted into RealRegs, and
possibly some spill code will have been inserted.

The heart of the register allocator works in four phases.

1.  (find_flow_edges) Calculate all the FEs for the code list.
    Return them not as a [FE], but implicitly, as a pair of
    Array Int [Int], being the successor and predecessor maps
    for instructions.

2.  (calc_liveness) Returns a FiniteMap FE RegSet.  For each
    FE, indicates the set of registers live on that FE.  Note
    that the set includes both RealRegs and VirtualRegs.  The
    former appear because the code could mention fixed register
    usages, and we need to take them into account from the start.

3.  (calc_live_range_sets) Invert the above mapping, giving a
    FiniteMap Reg FeSet, indicating, for each virtual and real
    reg mentioned in the code, which FEs it is live on.

4.  (calc_vreg_to_rreg_mapping) For virtual reg, try and find
    an allocatable real register for it.  Each real register has
    a "current commitment", indicating the set of FEs it is
    currently live on.  A virtual reg v can be assigned to
    real reg r iff v's live-fe-set does not intersect with r's
    current commitment fe-set.  If the assignment is made,
    v's live-fe-set is union'd into r's current commitment fe-set.
    There is also the minor restriction that v and r must be of
    the same register class (integer or floating).

    Once this mapping is established, we simply apply it to the
    input insns, and that's it.

    If no suitable real register can be found, the vreg is mapped
    to itself, and we deem allocation to have failed.  The partially
    allocated code is returned.  The higher echelons of the allocator
    (doGeneralAlloc and runRegAlloc) then cooperate to insert spill
    code and re-run allocation, until a successful allocation is found.