This optimisation actually make things a bit slower on average, as
measured by nofib. The example in #1136 in particular suffers from high
memory usage. Therefore we no longer do the optimisation.

Older GHCs can't parse OPTIONS_GHC.
This also changes the URL referenced for the -w options from
WorkingConventions#Warnings to CodingStyle#Warnings for the compiler
modules.

This has been a long-standing ToDo.

I added two new Core Lint checks in lintSingleBinding:

1. Check that the id's arity is equal to the
   number of arguments in its demand type, if it has a demand type
   at all (i.e., if demand analysis already happened).

2. Check that top-level or recursive binders aren't demanded.

This changes the internal representation of TickBoxes,
from
        Note (TickBox "module" n)  <expr>
into

        case tick<module,n> of
          _ -> <expr>

tick has type :: #State #World, when the module and tick numbe
are stored inside IdInfo.

Binary tick boxes change from

         Note (BinaryTickBox "module" t f) <expr>

into

          btick<module,t,f> <expr>

btick has type :: Bool -> Bool, with the module and tick number
stored inside IdInfo.

This patch is a start on removing import lists and generally tidying
up the top of each module.  In addition to removing import lists:

   - Change DATA.IOREF -> Data.IORef etc.
   - Change List -> Data.List etc.
   - Remove $Id$
   - Update copyrights
   - Re-order imports to put non-GHC imports last
   - Remove some unused and duplicate imports

Mon Sep 18 16:50:18 EDT 2006  Manuel M T Chakravarty <chak@cse.unsw.edu.au>
  * fixing record selectors
  Sun Aug  6 19:56:29 EDT 2006  Manuel M T Chakravarty <chak@cse.unsw.edu.au>
    * fixing record selectors
    Fri Jul 28 10:24:28 EDT 2006  kevind@bu.edu
    - Bad conflict in tcIfaceDataAlt, at a place where the monster patch had a 
      conflict, too.  I have no idea what the right code is.  -=chak
  NB (at time of 2nd merge): previous conflict resolution was fine

Fri Aug  4 15:11:01 EDT 2006  Manuel M T Chakravarty <chak@cse.unsw.edu.au>
  * Massive patch for the first months work adding System FC to GHC #1
  Broken up massive patch -=chak
  Original log message:  
  This is (sadly) all done in one patch to avoid Darcs bugs.
  It's not complete work... more FC stuff to come.  A compiler
  using just this patch will fail dismally.

Roman found that 
    loop :: STRef s a -> Int -> ST s Int
    loop ref n = case n of
                   0 -> return n
                   n -> loop ref (n-1)
wasn't eta-expanding nicely, despite the 'state hack'
(see Id.isStateHackType).  The reason was two-fold:

  a) a bug in CoreUtils.arityType (the Var case)

  b) the arity of a recursive function was not being
	exposed in its RHS (see commments with
	SimplEnv.addLetIdInfo

The commit fixes both.  

Add Id.idHasRules :: Id -> Bool, with the obvious semantics.
This patch makes sense by itself, but it's just a tidy-up.

Most of the other users of the fptools build system have migrated to
Cabal, and with the move to darcs we can now flatten the source tree
without losing history, so here goes.

The main change is that the ghc/ subdir is gone, and most of what it
contained is now at the top level.  The build system now makes no
pretense at being multi-project, it is just the GHC build system.

No doubt this will break many things, and there will be a period of
instability while we fix the dependencies.  A straightforward build
should work, but I haven't yet fixed binary/source distributions.
Changes to the Building Guide will follow, too.

This very large commit adds impredicativity to GHC, plus
numerous other small things.
  
*** WARNING: I have compiled all the libraries, and
***	     a stage-2 compiler, and everything seems
***	     fine.  But don't grab this patch if you 
***	     can't tolerate a hiccup if something is
***	     broken.
  
The big picture is this:

a) GHC handles impredicative polymorphism, as described in the
   "Boxy types: type inference for higher-rank types and
   impredicativity" paper

b) GHC handles GADTs in the new simplified (and very sligtly less
   epxrssive) way described in the
   "Simple unification-based type inference for GADTs" paper

  
But there are lots of smaller changes, and since it was pre-Darcs
they are not individually recorded.
  
Some things to watch out for:
  
c)   The story on lexically-scoped type variables has changed, as per
     my email.  I append the story below for completeness, but I 
     am still not happy with it, and it may change again.  In particular,
     the new story does not allow a pattern-bound scoped type variable
     to be wobbly, so (\(x::[a]) -> ...) is usually rejected.  This is
     more restrictive than before, and we might loosen up again.
  
d)   A consequence of adding impredicativity is that GHC is a bit less
     gung ho about converting automatically between
  	(ty1 -> forall a. ty2)    and    (forall a. ty1 -> ty2)
     In particular, you may need to eta-expand some functions to make
     typechecking work again.
   
     Furthermore, functions are now invariant in their argument types,
     rather than being contravariant.  Again, the main consequence is
     that you may occasionally need to eta-expand function arguments when
     using higher-rank polymorphism.
  

Please test, and let me know of any hiccups


Scoped type variables in GHC
~~~~~~~~~~~~~~~~~~~~~~~~~~~~
	January 2006

0) Terminology.
   
   A *pattern binding* is of the form
	pat = rhs

   A *function binding* is of the form
	f pat1 .. patn = rhs

   A binding of the formm
	var = rhs
   is treated as a (degenerate) *function binding*.


   A *declaration type signature* is a separate type signature for a
   let-bound or where-bound variable:
	f :: Int -> Int

   A *pattern type signature* is a signature in a pattern: 
	\(x::a) -> x
	f (x::a) = x

   A *result type signature* is a signature on the result of a
   function definition:
	f :: forall a. [a] -> a
	head (x:xs) :: a = x

   The form
	x :: a = rhs
   is treated as a (degnerate) function binding with a result
   type signature, not as a pattern binding.

1) The main invariants:

     A) A lexically-scoped type variable always names a (rigid)
 	type variable (not an arbitrary type).  THIS IS A CHANGE.
        Previously, a scoped type variable named an arbitrary *type*.

     B) A type signature always describes a rigid type (since
	its free (scoped) type variables name rigid type variables).
	This is also a change, a consequence of (A).

     C) Distinct lexically-scoped type variables name distinct
	rigid type variables.  This choice is open; 

2) Scoping

2(a) If a declaration type signature has an explicit forall, those type
   variables are brought into scope in the right hand side of the 
   corresponding binding (plus, for function bindings, the patterns on
   the LHS).  
	f :: forall a. a -> [a]
	f (x::a) = [x :: a, x]
   Both occurences of 'a' in the second line are bound by 
   the 'forall a' in the first line

   A declaration type signature *without* an explicit top-level forall
   is implicitly quantified over all the type variables that are
   mentioned in the type but not already in scope.  GHC's current
   rule is that this implicit quantification does *not* bring into scope
   any new scoped type variables.
	f :: a -> a
	f x = ...('a' is not in scope here)...
   This gives compatibility with Haskell 98

2(b) A pattern type signature implicitly brings into scope any type
   variables mentioned in the type that are not already into scope.
   These are called *pattern-bound type variables*.
	g :: a -> a -> [a]
	g (x::a) (y::a) = [y :: a, x]
   The pattern type signature (x::a) brings 'a' into scope.
   The 'a' in the pattern (y::a) is bound, as is the occurrence on 
   the RHS.  

   A pattern type siganture is the only way you can bring existentials 
   into scope.
	data T where
	  MkT :: forall a. a -> (a->Int) -> T

	f x = case x of
		MkT (x::a) f -> f (x::a)

2a) QUESTION
	class C a where
	  op :: forall b. b->a->a

	instance C (T p q) where
	  op = <rhs>
    Clearly p,q are in scope in <rhs>, but is 'b'?  Not at the moment.
    Nor can you add a type signature for op in the instance decl.
    You'd have to say this:
	instance C (T p q) where
	  op = let op' :: forall b. ...
	           op' = <rhs>
	       in op'

3) A pattern-bound type variable is allowed only if the pattern's
   expected type is rigid.  Otherwise we don't know exactly *which*
   skolem the scoped type variable should be bound to, and that means
   we can't do GADT refinement.  This is invariant (A), and it is a 
   big change from the current situation.

	f (x::a) = x	-- NO; pattern type is wobbly
	
	g1 :: b -> b
	g1 (x::b) = x	-- YES, because the pattern type is rigid

	g2 :: b -> b
	g2 (x::c) = x	-- YES, same reason

	h :: forall b. b -> b
	h (x::b) = x	-- YES, but the inner b is bound

	k :: forall b. b -> b
	k (x::c) = x	-- NO, it can't be both b and c

3a) You cannot give different names for the same type variable in the same scope
    (Invariant (C)):

	f1 :: p -> p -> p		-- NO; because 'a' and 'b' would be
	f1 (x::a) (y::b) = (x::a)	--     bound to the same type variable

	f2 :: p -> p -> p		-- OK; 'a' is bound to the type variable
	f2 (x::a) (y::a) = (x::a)	--     over which f2 is quantified
					-- NB: 'p' is not lexically scoped

	f3 :: forall p. p -> p -> p	-- NO: 'p' is now scoped, and is bound to
	f3 (x::a) (y::a) = (x::a)	--     to the same type varialble as 'a'

	f4 :: forall p. p -> p -> p	-- OK: 'p' is now scoped, and its occurences
	f4 (x::p) (y::p) = (x::p)	--     in the patterns are bound by the forall


3b) You can give a different name to the same type variable in different
    disjoint scopes, just as you can (if you want) give diferent names to 
    the same value parameter

	g :: a -> Bool -> Maybe a
	g (x::p) True  = Just x  :: Maybe p
	g (y::q) False = Nothing :: Maybe q

3c) Scoped type variables respect alpha renaming. For example, 
    function f2 from (3a) above could also be written:
	f2' :: p -> p -> p
	f2' (x::b) (y::b) = x::b
   where the scoped type variable is called 'b' instead of 'a'.


4) Result type signatures obey the same rules as pattern types signatures.
   In particular, they can bind a type variable only if the result type is rigid

	f x :: a = x	-- NO

	g :: b -> b
	g x :: b = x	-- YES; binds b in rhs

5) A *pattern type signature* in a *pattern binding* cannot bind a 
   scoped type variable

	(x::a, y) = ...		-- Legal only if 'a' is already in scope

   Reason: in type checking, the "expected type" of the LHS pattern is
   always wobbly, so we can't bind a rigid type variable.  (The exception
   would be for an existential type variable, but existentials are not
   allowed in pattern bindings either.)
 
   Even this is illegal
	f :: forall a. a -> a
	f x = let ((y::b)::a, z) = ... 
	      in 
   Here it looks as if 'b' might get a rigid binding; but you can't bind
   it to the same skolem as a.

6) Explicitly-forall'd type variables in the *declaration type signature(s)*
   for a *pattern binding* do not scope AT ALL.

	x :: forall a. a->a	  -- NO; the forall a does 
	Just (x::a->a) = Just id  --     not scope at all

	y :: forall a. a->a
	Just y = Just (id :: a->a)  -- NO; same reason

   THIS IS A CHANGE, but one I bet that very few people will notice.
   Here's why:

	strange :: forall b. (b->b,b->b)
	strange = (id,id)

	x1 :: forall a. a->a
	y1 :: forall b. b->b
	(x1,y1) = strange

    This is legal Haskell 98 (modulo the forall). If both 'a' and 'b'
    both scoped over the RHS, they'd get unified and so cannot stand
    for distinct type variables. One could *imagine* allowing this:
   
	x2 :: forall a. a->a
	y2 :: forall a. a->a
	(x2,y2) = strange

    using the very same type variable 'a' in both signatures, so that
    a single 'a' scopes over the RHS.  That seems defensible, but odd,
    because though there are two type signatures, they introduce just
    *one* scoped type variable, a.

7) Possible extension.  We might consider allowing
	\(x :: [ _ ]) -> <expr>
    where "_" is a wild card, to mean "x has type list of something", without
    naming the something.

Add support for UTF-8 source files

GHC finally has support for full Unicode in source files.  Source
files are now assumed to be UTF-8 encoded, and the full range of
Unicode characters can be used, with classifications recognised using
the implementation from Data.Char.  This incedentally means that only
the stage2 compiler will recognise Unicode in source files, because I
was too lazy to port the unicode classifier code into libcompat.

Additionally, the following synonyms for keywords are now recognised:

  forall symbol 	(U+2200)	forall
  right arrow   	(U+2192)	->
  left arrow   		(U+2190)	<-
  horizontal ellipsis 	(U+22EF)	..

there are probably more things we could add here.

This will break some source files if Latin-1 characters are being used.
In most cases this should result in a UTF-8 decoding error.  Later on
if we want to support more encodings (perhaps with a pragma to specify
the encoding), I plan to do it by recoding into UTF-8 before parsing.

Internally, there were some pretty big changes:

  - FastStrings are now stored in UTF-8

  - Z-encoding has been moved right to the back end.  Previously we
    used to Z-encode every identifier on the way in for simplicity,
    and only decode when we needed to show something to the user.
    Instead, we now keep every string in its UTF-8 encoding, and
    Z-encode right before printing it out.  To avoid Z-encoding the
    same string multiple times, the Z-encoding is cached inside the
    FastString the first time it is requested.

    This speeds up the compiler - I've measured some definite
    improvement in parsing at least, and I expect compilations overall
    to be faster too.  It also cleans up a lot of cruft from the
    OccName interface.  Z-encoding is nicely hidden inside the
    Outputable instance for Names & OccNames now.

  - StringBuffers are UTF-8 too, and are now represented as
    ForeignPtrs.

  - I've put together some test cases, not by any means exhaustive,
    but there are some interesting UTF-8 decoding error cases that
    aren't obvious.  Also, take a look at unicode001.hs for a demo.

Add record syntax for GADTs
	~~~~~~~~~~~~~~~~~~~~~~~~~~~

Atrijus Tang wanted to add record syntax for GADTs and existential
types, so he and I worked on it a bit at ICFP.  This commit is the
result.  Now you can say

 data T a where
  T1 { x :: a }           	 :: T [a]
  T2 { x :: a, y :: Int } 	 :: T [a]
  forall b. Show b =>
 	T3 { naughty :: b, ok :: Int } :: T Int
  T4 :: Eq a => a -> b -> T (a,b)

Here the constructors are declared using record syntax.

Still to come after this commit:
  - User manual documentation
  - More regression tests
  - Some missing cases in the parser (e.g. T3 won't parse)
Autrijus is going to do these.


Here's a quick summary of the rules.  (Atrijus is going to write
proper documentation shortly.)

Defnition: a 'vanilla' constructor has a type of the form
	forall a1..an. t1 -> ... -> tm -> T a1 ... an
No existentials, no context, nothing.  A constructor declared with
Haskell-98 syntax is vanilla by construction.  A constructor declared
with GADT-style syntax is vanilla iff its type looks like the above.
(In the latter case, the order of the type variables does not matter.)

* You can mix record syntax and non-record syntax in a single decl

* All constructors that share a common field 'x' must have the
  same result type (T [a] in the example).

* You can use field names without restriction in record construction
  and record pattern matching.

* Record *update* only works for data types that only have 'vanilla'
  constructors.

* Consider the field 'naughty', which uses a type variable that does
  not appear in the result type ('b' in the example).  You can use the
  field 'naughty' in pattern matching and construction, but NO
  SELECTOR function is generated for 'naughty'.  [An attempt to use
  'naughty' as a selector function will elicit a helpful error
  message.]

* Data types declared in GADT syntax cannot have a context. So this
is illegal:
	data (Monad m) => T a where
		  ....

* Constructors in GADT syntax can have a context (t.g. T3, T4 above)
  and that context is stored in the constructor and made available
  when the constructor is pattern-matched on.  WARNING: not competely
  implemented yet, but that's the plan.



Implementation notes
~~~~~~~~~~~~~~~~~~~~
- Data constructors (even vanilla ones) no longer share the type
  variables of their parent type constructor.

- HsDecls.ConDecl has changed quite a bit

- TyCons don't record the field labels and type any more (doesn't
  make sense for existential fields)

- GlobalIdDetails records which selectors are 'naughty', and hence
  don't have real code.

WARNING: this is a big commit.  You might want 
	to wait a few days before updating, in case I've 
	broken something.

	However, if any of the changes are what you wanted,
	please check it out and test!

This commit does three main things:

1. A re-organisation of the way that GHC handles bindings in HsSyn.
   This has been a bit of a mess for quite a while.  The key new
   types are

	-- Bindings for a let or where clause
	data HsLocalBinds id
	  = HsValBinds (HsValBinds id)
	  | HsIPBinds  (HsIPBinds id)
	  | EmptyLocalBinds

	-- Value bindings (not implicit parameters)
	data HsValBinds id
	  = ValBindsIn  -- Before typechecking
		(LHsBinds id) [LSig id]	-- Not dependency analysed
					-- Recursive by default

	  | ValBindsOut	-- After typechecking
		[(RecFlag, LHsBinds id)]-- Dependency analysed

2. Implement Mark Jones's idea of increasing polymoprhism
   by using type signatures to cut the strongly-connected components
   of a recursive group.  As a consequence, GHC no longer insists
   on the contexts of the type signatures of a recursive group
   being identical.

   This drove a significant change: the renamer no longer does dependency
   analysis.  Instead, it attaches a free-variable set to each binding,
   so that the type checker can do the dep anal.  Reason: the typechecker
   needs to do *two* analyses:
	one to find the true mutually-recursive groups
		(which we need so we can build the right CoreSyn)
	one to find the groups in which to typecheck, taking
		account of type signatures

3. Implement non-ground SPECIALISE pragmas, as promised, and as
   requested by Remi and Ross.  Certainly, this should fix the 
   current problem with GHC, namely that if you have
	g :: Eq a => a -> b -> b
   then you can now specialise thus
	SPECIALISE g :: Int -> b -> b
    (This didn't use to work.)

   However, it goes further than that.  For example:
	f :: (Eq a, Ix b) => a -> b -> b
   then you can make a partial specialisation
	SPECIALISE f :: (Eq a) => a -> Int -> Int

    In principle, you can specialise f to *any* type that is
    "less polymorphic" (in the sense of subsumption) than f's 
    actual type.  Such as
	SPECIALISE f :: Eq a => [a] -> Int -> Int
    But I haven't tested that.

    I implemented this by doing the specialisation in the typechecker
    and desugarer, rather than leaving around the strange SpecPragmaIds,
    for the specialiser to find.  Indeed, SpecPragmaIds have vanished 
    altogether (hooray).

    Pragmas in general are handled more tidily.  There's a new
    data type HsBinds.Prag, which lives in an AbsBinds, and carries
    pragma info from the typechecker to the desugarer.


Smaller things

- The loop in the renamer goes via RnExpr, instead of RnSource.
  (That makes it more like the type checker.)

- I fixed the thing that was causing 'check_tc' warnings to be 
  emitted.

Re-implement GHCi's :info and :browse commands in terms of TyThings
rather than IfaceSyn.

The GHC API now exposes its internal types for Haskell entities:
TyCons, Classes, DataCons, Ids and Instances (collectively known as
TyThings), so we can inspect these directly to pretty-print
information about an entity.  Previously the internal representations
were converted to IfaceSyn for passing to InteractiveUI, but we can
now remove that code.

Some of the new code comes via Visual Haskell, but I've changed it
around a lot to fix various dark corners and properly print things
like GADTs.

The pretty-printing interfaces for TyThings are exposed by a new
module PprTyThing, which is implemented purely in terms of the GHC API
(and is probably a good source of sample code).  Visual Haskell should
be able to use the functions exported by this module directly.

Lots of new goodies are exported by the GHC module, mainly for
inspecting TyThings.

This big commit does several things at once (aeroplane hacking)
which change the format of interface files.  

	So you'll need to recompile your libraries!

1. The "stupid theta" of a newtype declaration
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Retain the "stupid theta" in a newtype declaration.
For some reason this was being discarded, and putting it
back in meant changing TyCon and IfaceSyn slightly.
   

2. Overlap flags travel with the instance
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Arrange that the ability to support overlap and incoherence
is a property of the *instance declaration* rather than the
module that imports the instance decl.  This allows a library
writer to define overlapping instance decls without the
library client having to know.  

The implementation is that in an Instance we store the
overlap flag, and preseve that across interface files


3. Nuke the "instnce pool" and "rule pool"
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
A major tidy-up and simplification of the way that instances
and rules are sucked in from interface files.  Up till now
an instance decl has been held in a "pool" until its "gates" 
(a set of Names) are in play, when the instance is typechecked
and added to the InstEnv in the ExternalPackageState.  
This is complicated and error-prone; it's easy to suck in 
too few (and miss an instance) or too many (and thereby be
forced to suck in its type constructors, etc).

Now, as we load an instance from an interface files, we 
put it straight in the InstEnv... but the Instance we put in
the InstEnv has some Names (the "rough-match" names) that 
can be used on lookup to say "this Instance can't match".
The detailed dfun is only read lazily, and the rough-match
thing meansn it is'nt poked on until it has a chance of
being needed.

This simply continues the successful idea for Ids, whereby
they are loaded straightaway into the TypeEnv, but their
TyThing is a lazy thunk, not poked on until the thing is looked
up.

Just the same idea applies to Rules.

On the way, I made CoreRule and Instance into full-blown records
with lots of info, with the same kind of key status as TyCon or 
DataCon or Class.  And got rid of IdCoreRule altogether.   
It's all much more solid and uniform, but it meant touching
a *lot* of modules.


4. Allow instance decls in hs-boot files
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Allowing instance decls in hs-boot files is jolly useful, becuase
in a big mutually-recursive bunch of data types, you want to give
the instances with the data type declarations.  To achieve this

* The hs-boot file makes a provisional name for the dict-fun, something
  like $fx9.

* When checking the "mother module", we check that the instance
  declarations line up (by type) and generate bindings for the 
  boot dfuns, such as
	$fx9 = $f2
  where $f2 is the dfun generated by the mother module

* In doing this I decided that it's cleaner to have DFunIds get their
  final External Name at birth.  To do that they need a stable OccName,
  so I have an integer-valued dfun-name-supply in the TcM monad.
  That keeps it simple.

This feature is hardly tested yet.


5. Tidy up tidying, and Iface file generation
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
main/TidyPgm now has two entry points:

  simpleTidyPgm is for hi-boot files, when typechecking only
  (not yet implemented), and potentially when compiling without -O.
  It ignores the bindings, and generates a nice small TypeEnv.

  optTidyPgm is the normal case: compiling with -O.  It generates a
  TypeEnv rich in IdInfo

MkIface.mkIface now only generates a ModIface.  A separate
procedure, MkIface.writeIfaceFile, writes the file out to disk.

Final wibbles, I hope

Flags cleanup.

Basically the purpose of this commit is to move more of the compiler's
global state into DynFlags, which is moving in the direction we need
to go for the GHC API which can have multiple active sessions
supported by a single GHC instance.

Before:

$ grep 'global_var' */*hs | wc -l
     78

After:

$ grep 'global_var' */*hs | wc -l
     27

Well, it's an improvement.  Most of what's left won't really affect
our ability to host multiple sessions.

Lots of static flags have become dynamic flags (yay!).  Notably lots
of flags that we used to think of as "driver" flags, like -I and -L,
are now dynamic.  The most notable static flags left behind are the
"way" flags, eg. -prof.  It would be nice to fix this, but it isn't
urgent.

On the way, lots of cleanup has happened.  Everything related to
static and dynamic flags lives in StaticFlags and DynFlags
respectively, and they share a common command-line parser library in
CmdLineParser.  The flags related to modes (--makde, --interactive
etc.) are now private to the front end: in fact private to Main
itself, for now.

---------------------------------------------
Type signatures are no longer instantiated with skolem constants
	---------------------------------------------

	Merge to STABLE

Consider

  p :: a
  q :: b
  (p,q,r) = (r,r,p)

Here, 'a' and 'b' end up being the same, because they are both bound
to the type for 'r', which is just a meta type variable.  So 'a' and 'b'
can't be skolems.

Sigh.  This commit goes back to an earlier way of doing things, by
arranging that type signatures get instantiated with *meta* type
variables; then at the end we must check that they have not been
unified with types, nor with each other.

This is a real bore.  I had to do quite a bit of related fiddling around
to make error messages come out right.  Improved one or two.

Also a small unrelated fix to make
	:i (:+)
print with parens in ghci.  Sorry this got mixed up in the same commit.

Reset the export flag for the new bindings in LiberateCase

------------------------------------------
	Keep-alive set and Template Haskell quotes
	------------------------------------------

a) Template Haskell quotes should be able to mention top-leve
   things without resorting to lifting.  Example

	module Foo( foo ) where
	  f x = x
	  foo = [| f 4 |]

   Here the reference to 'f' is ok; no need to 'lift' it.
   The relevant changes are in TcExpr.tcId

b) However, we must take care not to discard the binding for f,
   so we add it to the 'keep-alive' set for the module.  I've
   now made this into (another) mutable bucket, tcg_keep, 
   in the TcGblEnv

c) That in turn led me to look at the handling of orphan rules;
   as a result I made IdCoreRule into its own data type, which
   has simle but non-local ramifications

------------------------------------
	Add Generalised Algebraic Data Types
	------------------------------------

This rather big commit adds support for GADTs.  For example,

    data Term a where
 	  Lit :: Int -> Term Int
	  App :: Term (a->b) -> Term a -> Term b
	  If  :: Term Bool -> Term a -> Term a
	  ..etc..

    eval :: Term a -> a
    eval (Lit i) = i
    eval (App a b) = eval a (eval b)
    eval (If p q r) | eval p    = eval q
    		    | otherwise = eval r


Lots and lots of of related changes throughout the compiler to make
this fit nicely.

One important change, only loosely related to GADTs, is that skolem
constants in the typechecker are genuinely immutable and constant, so
we often get better error messages from the type checker.  See
TcType.TcTyVarDetails.

There's a new module types/Unify.lhs, which has purely-functional
unification and matching for Type. This is used both in the typechecker
(for type refinement of GADTs) and in Core Lint (also for type refinement).

Merge backend-hacking-branch onto HEAD.  Yay!

----------------------------------------
     1. Make primOpIsCheap do something sensible
     2. Make the state hack work better
     ----------------------------------------

1.  In March 2001, we changed primOpIsCheap to
	primOpIsCheap op = False
thereby making *no* primops seem cheap.  But this killed eta
expansion on case (x ==# y) of True -> \s -> ...
which is bad.  In particular a loop like
  doLoop n = loop 0
    where
      loop i | i == n    = return ()
             | otherwise = bar i >> loop (i+1)
allocated a closure every time round because it didn't eta expand.

The problem that made us set primOpIsCheap to False was
		let x = a +# b *# c in x +# x
where we don't want to inline x. But primopIsCheap doesn't control
that (it's exprIsDupable that does) so the problem doesn't occur
even if primOpIsCheap sometimes says 'True'.  I think that perhaps
something changed since March 2001.


2.  Consider this
	case x of
	  True  -> \(s:RealWorld) -> e
	  False -> foo
where foo has arity 1.  If we are using the "state hack" we want to
eta expand here.  This commit fixes arityType in the Var case (for foo)
to take account of foo's type.

Also add -fno-state-hack to the static flags, to allow the state hack to
be switched off.

Add a flag -fno-state-hack, which switches off the "state hack".

It's claims that every function over realWorldStatePrimTy is a
one-shot function.  This is pretty true in practice, and makes a big
difference.  For example, consider
	a `thenST` \ r -> ...E...
The early full laziness pass, if it doesn't know that r is one-shot
will pull out E (let's say it doesn't mention r) to give
	let lvl = E in a `thenST` \ r -> ...lvl...
When `thenST` gets inlined, we end up with
	let lvl = E in \s -> case a s of (r, s') -> ...lvl...
and we don't re-inline E.

Fix globaliseId wibble which was killing ghci

----------------------------
        Re-do kind inference (again)
	----------------------------

   [WARNING: interface file binary representation has
   (as usual) changed slightly; recompile your libraries!]

Inspired by the lambda-cube, for some time GHC has used
	type Kind = Type
That is, kinds were represented by the same data type as types.

But GHC also supports unboxed types and unboxed tuples, and these
complicate the kind system by requiring a sub-kind relationship.
Notably, an unboxed tuple is acceptable as the *result* of a
function but not as an *argument*.  So we have the following setup:

		 ?
		/ \
	       /   \
	      ??   (#)
	     /  \
            *   #

where	*    [LiftedTypeKind]   means a lifted type
	#    [UnliftedTypeKind] means an unlifted type
	(#)  [UbxTupleKind]     means unboxed tuple
	??   [ArgTypeKind]      is the lub of *,#
	?    [OpenTypeKind]	means any type at all

In particular:

  error :: forall a:?. String -> a
  (->)  :: ?? -> ? -> *
  (\(x::t) -> ...)	Here t::?? (i.e. not unboxed tuple)

All this has beome rather difficult to accommodate with Kind=Type, so this
commit splits the two.

  * Kind is a distinct type, defined in types/Kind.lhs

  * IfaceType.IfaceKind disappears: we just re-use Kind.Kind

  * TcUnify.unifyKind is a distinct unifier for kinds

  * TyCon no longer needs KindCon and SuperKindCon variants

  * TcUnify.zapExpectedType takes an expected Kind now, so that
    in TcPat.tcMonoPatBndr we can express that the bound variable
    must have an argTypeKind (??).

The big change is really that kind inference is much more systematic and
well behaved.  In particular, a kind variable can unify only with a
"simple kind", which is built from * and (->).  This deals neatly
with awkward questions about how we can combine sub-kinding with type
inference.

Lots of small consequential changes, especially to the kind-checking
plumbing in TcTyClsDecls.  (We played a bit fast and loose before, and
now we have to be more honest, in particular about how kind inference
works for type synonyms.  They can have kinds like (* -> #), so

This cures two long-standing SourceForge bugs

* 753777 (tcfail115.hs), which used erroneously to pass,
  but crashed in the code generator
      type T a = Int -> (# Int, Int #)
      f :: T a -> T a
      f t = \x -> case t x of r -> r

* 753780 (tc167.hs), which used erroneously to fail
      f :: (->) Int# Int#


Still, the result is not entirely satisfactory.  In particular

* The error message from tcfail115 is pretty obscure

* SourceForge bug 807249 (Instance match failure on openTypeKind)
  is not fixed.  Alas.

Add idIsFrom

-------------------------
		GHC heart/lung transplant
		-------------------------

This major commit changes the way that GHC deals with importing
types and functions defined in other modules, during renaming and
typechecking.  On the way I've changed or cleaned up numerous other
things, including many that I probably fail to mention here.

Major benefit: GHC should suck in many fewer interface files when
compiling (esp with -O).  (You can see this with -ddump-rn-stats.)

It's also some 1500 lines of code shorter than before.

**	So expect bugs!  I can do a 3-stage bootstrap, and run
**	the test suite, but you may be doing stuff I havn't tested.
** 	Don't update if you are relying on a working HEAD.


In particular, (a) External Core and (b) GHCi are very little tested.

	But please, please DO test this version!


	------------------------
		Big things
	------------------------

Interface files, version control, and importing declarations
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
* There is a totally new data type for stuff that lives in interface files:
	Original names			IfaceType.IfaceExtName
	Types				IfaceType.IfaceType
	Declarations (type,class,id)	IfaceSyn.IfaceDecl
	Unfoldings			IfaceSyn.IfaceExpr
  (Previously we used HsSyn for type/class decls, and UfExpr for unfoldings.)
  The new data types are in iface/IfaceType and iface/IfaceSyn.  They are
  all instances of Binary, so they can be written into interface files.
  Previous engronkulation concering the binary instance of RdrName has
  gone away -- RdrName is not an instance of Binary any more.  Nor does
  Binary.lhs need to know about the ``current module'' which it used to,
  which made it specialised to GHC.

  A good feature of this is that the type checker for source code doesn't
  need to worry about the possibility that we might be typechecking interface
  file stuff.  Nor does it need to do renaming; we can typecheck direct from
  IfaceSyn, saving a whole pass (module TcIface)

* Stuff from interface files is sucked in *lazily*, rather than being eagerly
  sucked in by the renamer. Instead, we use unsafeInterleaveIO to capture
  a thunk for the unfolding of an imported function (say).  If that unfolding
  is every pulled on, TcIface will scramble over the unfolding, which may
  in turn pull in the interface files of things mentioned in the unfolding.

  The External Package State is held in a mutable variable so that it
  can be side-effected by this lazy-sucking-in process (which may happen
  way later, e.g. when the simplifier runs).   In effect, the EPS is a kind
  of lazy memo table, filled in as we suck things in.  Or you could think
  of it as a global symbol table, populated on demand.

* This lazy sucking is very cool, but it can lead to truly awful bugs. The
  intent is that updates to the symbol table happen atomically, but very bad
  things happen if you read the variable for the table, and then force a
  thunk which updates the table.  Updates can get lost that way. I regret
  this subtlety.

  One example of the way it showed up is that the top level of TidyPgm
  (which updates the global name cache) to be much more disciplined about
  those updates, since TidyPgm may itself force thunks which allocate new
  names.

* Version numbering in interface files has changed completely, fixing
  one major bug with ghc --make.  Previously, the version of A.f changed
  only if A.f's type and unfolding was textually different.  That missed
  changes to things that A.f's unfolding mentions; which was fixed by
  eagerly sucking in all of those things, and listing them in the module's
  usage list.  But that didn't work with --make, because they might have
  been already sucked in.

  Now, A.f's version changes if anything reachable from A.f (via interface
  files) changes.  A module with unchanged source code needs recompiling
  only if the versions of any of its free variables changes. [This isn't
  quite right for dictionary functions and rules, which aren't mentioned
  explicitly in the source.  There are extensive comments in module MkIface,
  where all version-handling stuff is done.]

* We don't need equality on HsDecls any more (because they aren't used in
  interface files).  Instead we have a specialised equality for IfaceSyn
  (eqIfDecl etc), which uses IfaceEq instead of Bool as its result type.
  See notes in IfaceSyn.

* The horrid bit of the renamer that tried to predict what instance decls
  would be needed has gone entirely.  Instead, the type checker simply
  sucks in whatever instance decls it needs, when it needs them.  Easy!

  Similarly, no need for 'implicitModuleFVs' and 'implicitTemplateHaskellFVs'
  etc.  Hooray!


Types and type checking
~~~~~~~~~~~~~~~~~~~~~~~
* Kind-checking of types is far far tidier (new module TcHsTypes replaces
  the badly-named TcMonoType).  Strangely, this was one of my
  original goals, because the kind check for types is the Right Place to
  do type splicing, but it just didn't fit there before.

* There's a new representation for newtypes in TypeRep.lhs.  Previously
  they were represented using "SourceTypes" which was a funny compromise.
  Now they have their own constructor in the Type datatype.  SourceType
  has turned back into PredType, which is what it used to be.

* Instance decl overlap checking done lazily.  Consider
	instance C Int b
	instance C a Int
  These were rejected before as overlapping, because when seeking
  (C Int Int) one couldn't tell which to use.  But there's no problem when
  seeking (C Bool Int); it can only be the second.

  So instead of checking for overlap when adding a new instance declaration,
  we check for overlap when looking up an Inst.  If we find more than one
  matching instance, we see if any of the candidates dominates the others
  (in the sense of being a substitution instance of all the others);
  and only if not do we report an error.



	------------------------
	     Medium things
	------------------------

* The TcRn monad is generalised a bit further.  It's now based on utils/IOEnv.lhs,
  the IO monad with an environment.  The desugarer uses the monad too,
  so that anything it needs can get faulted in nicely.

* Reduce the number of wired-in things; in particular Word and Integer
  are no longer wired in.  The latter required HsLit.HsInteger to get a
  Type argument.  The 'derivable type classes' data types (:+:, :*: etc)
  are not wired in any more either (see stuff about derivable type classes
  below).

* The PersistentComilerState is now held in a mutable variable
  in the HscEnv.  Previously (a) it was passed to and then returned by
  many top-level functions, which was painful; (b) it was invariably
  accompanied by the HscEnv.  This change tidies up top-level plumbing
  without changing anything important.

* Derivable type classes are treated much more like 'deriving' clauses.
  Previously, the Ids for the to/from functions lived inside the TyCon,
  but now the TyCon simply records their existence (with a simple boolean).
  Anyone who wants to use them must look them up in the environment.

  This in turn makes it easy to generate the to/from functions (done
  in types/Generics) using HsSyn (like TcGenDeriv for ordinary derivings)
  instead of CoreSyn, which in turn means that (a) we don't have to figure
  out all the type arguments etc; and (b) it'll be type-checked for us.
  Generally, the task of generating the code has become easier, which is
  good for Manuel, who wants to make it more sophisticated.

* A Name now says what its "parent" is. For example, the parent of a data
  constructor is its type constructor; the parent of a class op is its
  class.  This relationship corresponds exactly to the Avail data type;
  there may be other places we can exploit it.  (I made the change so that
  version comparison in interface files would be a bit easier; but in
  fact it tided up other things here and there (see calls to
  Name.nameParent).  For example, the declaration pool, of declararations
  read from interface files, but not yet used, is now keyed only by the 'main'
  name of the declaration, not the subordinate names.

* New types OccEnv and OccSet, with the usual operations.
  OccNames can be efficiently compared, because they have uniques, thanks
  to the hashing implementation of FastStrings.

* The GlobalRdrEnv is now keyed by OccName rather than RdrName.  Not only
  does this halve the size of the env (because we don't need both qualified
  and unqualified versions in the env), but it's also more efficient because
  we can use a UniqFM instead of a FiniteMap.

  Consequential changes to Provenance, which has moved to RdrName.

* External Core remains a bit of a hack, as it was before, done with a mixture
  of HsDecls (so that recursiveness and argument variance is still inferred),
  and IfaceExprs (for value declarations).  It's not thoroughly tested.


	------------------------
	     Minor things
	------------------------

* DataCon fields dcWorkId, dcWrapId combined into a single field
  dcIds, that is explicit about whether the data con is a newtype or not.
  MkId.mkDataConWorkId and mkDataConWrapId are similarly combined into
  MkId.mkDataConIds

* Choosing the boxing strategy is done for *source* type decls only, and
  hence is now in TcTyDecls, not DataCon.

* WiredIn names are distinguished by their n_sort field, not by their location,
  which was rather strange

* Define Maybes.mapCatMaybes :: (a -> Maybe b) -> [a] -> [b]
  and use it here and there

* Much better pretty-printing of interface files (--show-iface)

Many, many other small things.


	------------------------
	     File changes
	------------------------
* New iface/ subdirectory
* Much of RnEnv has moved to iface/IfaceEnv
* MkIface and BinIface have moved from main/ to iface/
* types/Variance has been absorbed into typecheck/TcTyDecls
* RnHiFiles and RnIfaces have vanished entirely.  Their
  work is done by iface/LoadIface
* hsSyn/HsCore has gone, replaced by iface/IfaceSyn
* typecheck/TcIfaceSig has gone, replaced by iface/TcIface
* typecheck/TcMonoType has been renamed to typecheck/TcHsType
* basicTypes/Var.hi-boot and basicTypes/Generics.hi-boot have gone altogether

#ifdef some more code that belongs to the old strictness analyser.

It turns out we were still compiling the Demand and SaLib modules,
which aren't required unless OLD_STRICTNESS is on (do we still need
OLD_STRICTNESS?).

----------------------------------
      Fix a long-standing egregious CorePrep bug
	----------------------------------

	**** Merge to Stable branch ****

CorePrep was eta-reducing

	\x -> (# a,x #)
to
	(#,#) a

which is utterly wrong.  Easily fixed though.

A round of space-leak fixing.

  - re-instate zapping of the PersistentCompilerState at various
    points during the compilation cycle in HscMain.  This affects
    one-shot compilation only, since in this mode the information
    collected in the PCS is not required after creating the final
    interface file.

  - Unravel the recursive dependency between MkIface and
    CoreTidy/CoreToStg.  Previously the CafInfo for each binding was
    calculated by CoreToStg, and fed back into the IdInfo of the Ids
    generated by CoreTidy (an earlier pass).  MkIface then took this
    IdInfo and the bindings from CoreTidy to generate the interface;
    but it couldn't do this until *after* CoreToStg, because the CafInfo
    hadn't been calculated yet.  The result was that the CoreTidy
    output lived until after CoreToStg, and at the same time as the
    CorePrep and STG syntax, which is wasted space, not to mention
    the complexity and general ugliness in HscMain.

    So now we calculate CafInfo directly in CoreTidy.  The downside is
    that we have to predict what CorePrep is going to do to the
    bindings so we can tell what will turn into a CAF later, but it's
    no worse than before (it turned out that we were doing this
    prediction before in CoreToStg anyhow).

  - The typechecker lazilly typechecks unfoldings.  It turns out that
    this is a good idea from a performance perspective, but it also
    means that it must hang on to all the information it needs to
    do the typechecking.  Previously this meant holding on to the
    whole of the typechecker's environment, which includes all sorts
    of stuff which isn't necessary to typecheck unfoldings.  By paring
    down the environment captured by the lazy unfoldings, we can
    save quite a bit of space in the phases after typechecking.

-------------------------------------
	 	Two minor wibbles
	-------------------------------------


1.  Make the generic toT/fromT Ids for "generic derived classes" into
    proper ImplicitIds, with their own GlobalIdDetails. This makes it
    easier to identify them.  (The lack of this showed up as a bug
    when I made an apparently-innocuous other change.)

2.  Distinguish ClassOpIds from RecordSelIds in their GlobalIdDetails.
    They are treated differently here and there, so I made this change
    as part of (1)

3.  Ensure that a declaration quotation [d| ... |] does not have a
    permanent effect on the instance environment. (A TH fix.)