Fix hi-boot interface-finding code

Significant clean-up of the handling of hi-boot files. 
Previously, when compling A.hs, we loaded A.hi-boot, and
it went into the External Package Table.  It was strange
but it worked.  This tidy up stops it going anywhere;
it's just read in, and typechecked into a ModDetails.

All this was on the way to improving the handling of
instances in hs-boot files, something Chris Ryder wanted.
I think they work quite sensibly now.  

If I've got all this right (have not had a chance to
fully test it) we can merge it into STABLE.

Flags cleanup.

Basically the purpose of this commit is to move more of the compiler's
global state into DynFlags, which is moving in the direction we need
to go for the GHC API which can have multiple active sessions
supported by a single GHC instance.

Before:

$ grep 'global_var' */*hs | wc -l
     78

After:

$ grep 'global_var' */*hs | wc -l
     27

Well, it's an improvement.  Most of what's left won't really affect
our ability to host multiple sessions.

Lots of static flags have become dynamic flags (yay!).  Notably lots
of flags that we used to think of as "driver" flags, like -I and -L,
are now dynamic.  The most notable static flags left behind are the
"way" flags, eg. -prof.  It would be nice to fix this, but it isn't
urgent.

On the way, lots of cleanup has happened.  Everything related to
static and dynamic flags lives in StaticFlags and DynFlags
respectively, and they share a common command-line parser library in
CmdLineParser.  The flags related to modes (--makde, --interactive
etc.) are now private to the front end: in fact private to Main
itself, for now.

Add forall-hoisting to TcIface; see comments with mkIfTcAPp
	Fixes Sourceforge bug 1146068
	tc191 tests

This fix is temporary, until we get rid of forall-hoisting 
altogether

Fix a recompilation bug caused by the fact that typecheckIface wasn't
going via loadDecl to create the binders properly.  The fix actually
results in slightly cleaner code.

Arrange that when seeking instance decls in GHCi, in response
to a :info command, we only print ones whose types are in scope
unqualified.  This eliminates an alarmingly long list when
simply typing ':info Show', say.

On the way, I reorganised a bit.  GHCi printing happens by
converting a TyThing to an IfaceDecl, and printing that.
I now arrange to generate unqualifed IfaceExtNames directly
during this conversion, based on what is in scope.  Previously
it was done during the pretty-printing part via the UserStyle.
But this is nicer.

Comments

Fix syntax error

------------------------
    Reorganisation of hi-boot files
  	------------------------

The main point of this commit is to arrange that in the Compilation
Manager's dependendency graph, hi-boot files are proper nodes. This
is important to make sure that we compile everything in the right
order.  It's a step towards hs-boot files.

* The fundamental change is that CompManager.ModSummary has a new
  field, ms_boot :: IsBootInterface

  I also tided up CompManager a bit.  No change to the Basic Plan.

  ModSummary is now exported abstractly from CompManager (was concrete)

* Hi-boot files now have import declarations.  The idea is they are
  compulsory, so that the dependency analyser can find them

* I changed an invariant: the Compilation Manager used to ensure that
  hscMain was given a HomePackageTable only for the modules 'below' the
  one being compiled.  This was really only important for instances and
  rules, and it was a bit inconvenient.  So I moved the filter to the
  compiler itself: see HscTypes.hptInstances and hptRules.

* Module Packages.hs now defines
    data PackageIdH
    = HomePackage 		-- The "home" package is the package
 				-- curently being compiled
    | ExtPackage PackageId	-- An "external" package is any other package

   It was just a Maybe type before, so this makes it a bit clearer.

* I tried to add a bit better location info to the IfM monad, so that
  errors in interfaces come with a slightly more helpful error message.
  See the if_loc field in TcRnTypes --- and follow-on consequences

* Changed Either to Maybes.MaybeErr in a couple of places (more perspicuous)

----------------------------------------
	     Add more scoped type variables
	----------------------------------------

Now the top-level forall'd variables of a type signature scope
over the right hand side of that function.

	f :: a -> a
	f x = ....

The type variable 'a' is in scope in the RHS, and in f's patterns.

It's implied by -fglasgow-exts, but can also be switched off independently
using -fscoped-type-variables (and the -fno variant)

----------------------------------------
     New Core invariant: keep case alternatives in sorted order
	----------------------------------------

We now keep the alternatives of a Case in the Core language in sorted
order.  Sorted, that is,
	by constructor tag	for DataAlt
	by literal		for LitAlt

The main reason is that it makes matching and equality testing more robust.
But in fact some lines of code vanished from SimplUtils.mkAlts.


WARNING: no change to interface file formats, but you'll need to recompile
your libraries so that they generate interface files that respect the
invariant.

Further integration with the new package story.  GHC now supports
pretty much everything in the package proposal.

  - GHC now works in terms of PackageIds (<pkg>-<version>) rather than
    just package names.  You can still specify package names without
    versions on the command line, as long as the name is unambiguous.

  - GHC understands hidden/exposed modules in a package, and will refuse
    to import a hidden module.  Also, the hidden/eposed status of packages
    is taken into account.

  - I had to remove the old package syntax from ghc-pkg, backwards
    compatibility isn't really practical.

  - All the package.conf.in files have been rewritten in the new syntax,
    and contain a complete list of modules in the package.  I've set all
    the versions to 1.0 for now - please check your package(s) and fix the
    version number & other info appropriately.

  - New options:

	-hide-package P    sets the expose flag on package P to False
	-ignore-package P  unregisters P for this compilation

	For comparison, -package P sets the expose flag on package P
        to True, and also causes P to be linked in eagerly.

        -package-name is no longer officially supported.  Unofficially, it's
	a synonym for -ignore-package, which has more or less the same effect
	as -package-name used to.

	Note that a package may be hidden and yet still be linked into
	the program, by virtue of being a dependency of some other package.
	To completely remove a package from the compiler's internal database,
        use -ignore-package.

	The compiler will complain if any two packages in the
        transitive closure of exposed packages contain the same
        module.

	You *must* use -ignore-package P when compiling modules for
        package P, if package P (or an older version of P) is already
        registered.  The compiler will helpfully complain if you don't.
	The fptools build system does this.

   - Note: the Cabal library won't work yet.  It still thinks GHC uses
     the old package config syntax.

Internal changes/cleanups:

   - The ModuleName type has gone away.  Modules are now just (a
     newtype of) FastStrings, and don't contain any package information.
     All the package-related knowledge is in DynFlags, which is passed
     down to where it is needed.

   - DynFlags manipulation has been cleaned up somewhat: there are no
     global variables holding DynFlags any more, instead the DynFlags
     are passed around properly.

   - There are a few less global variables in GHC.  Lots more are
     scheduled for removal.

   - -i is now a dynamic flag, as are all the package-related flags (but
     using them in {-# OPTIONS #-} is Officially Not Recommended).

   - make -j now appears to work under fptools/libraries/.  Probably
     wouldn't take much to get it working for a whole build.

------------------------------------------
	Keep-alive set and Template Haskell quotes
	------------------------------------------

a) Template Haskell quotes should be able to mention top-leve
   things without resorting to lifting.  Example

	module Foo( foo ) where
	  f x = x
	  foo = [| f 4 |]

   Here the reference to 'f' is ok; no need to 'lift' it.
   The relevant changes are in TcExpr.tcId

b) However, we must take care not to discard the binding for f,
   so we add it to the 'keep-alive' set for the module.  I've
   now made this into (another) mutable bucket, tcg_keep, 
   in the TcGblEnv

c) That in turn led me to look at the handling of orphan rules;
   as a result I made IdCoreRule into its own data type, which
   has simle but non-local ramifications

------------------------------------
	Add Generalised Algebraic Data Types
	------------------------------------

This rather big commit adds support for GADTs.  For example,

    data Term a where
 	  Lit :: Int -> Term Int
	  App :: Term (a->b) -> Term a -> Term b
	  If  :: Term Bool -> Term a -> Term a
	  ..etc..

    eval :: Term a -> a
    eval (Lit i) = i
    eval (App a b) = eval a (eval b)
    eval (If p q r) | eval p    = eval q
    		    | otherwise = eval r


Lots and lots of of related changes throughout the compiler to make
this fit nicely.

One important change, only loosely related to GADTs, is that skolem
constants in the typechecker are genuinely immutable and constant, so
we often get better error messages from the type checker.  See
TcType.TcTyVarDetails.

There's a new module types/Unify.lhs, which has purely-functional
unification and matching for Type. This is used both in the typechecker
(for type refinement of GADTs) and in Core Lint (also for type refinement).

-------------------------------
	Add instance information to :i
 	Get rid of the DeclPool
	-------------------------------


1.  Add instance information to :info command.  GHCi now prints out
    which instances a type or class belongs to, when you use :i

2.  Tidy up printing of unqualified names in user output.
    Previously Outputable.PrintUnqualified was
	type PrintUnqualified = Name -> Bool
    but it's now
	type PrintUnqualified = ModuleName -> OccName -> Bool
    This turns out to be tidier even for Names, and it's now also usable
    when printing IfaceSyn stuff in GHCi, eliminating a grevious hack.

3.  On the way to doing this, Simon M had the great idea that we could
    get rid of the DeclPool holding pen, which held declarations read from
    interface files but not yet type-checked.   We do this by eagerly
    populating the TypeEnv with thunks what, when poked, do the type
    checking.   This is just a logical continuation of lazy import
    mechanism we've now had for some while.

The InstPool and RulePool still exist, but I plan to get rid of them in
the same way.  The new scheme does mean that more rules get sucked in than
before, because previously the TypeEnv was used to mean "this thing was needed"
and hence to control which rules were sucked in.  But now the TypeEnv is
populated more eagerly => more rules get sucked in.  However this problem
will go away when I get rid of the Inst and Rule pools.

I should have kept these changes separate, but I didn't.  Change (1)
affects mainly
	TcRnDriver, HscMain, CompMan, InteractiveUI
whereas change (3) is more wide ranging.

-----------------------------------------------
       Record whether data constructors are declared infix
	-----------------------------------------------

This allows us to generate the InfixC form in Template Hasekll.
And for 'deriving' Read and Show, we now read and parse the infix
form iff the constructor was declared infix, rather than just if
it does not have the default fixity (as before).

IfaceSyn changes slightly, so that IfaceConDecl can record their
fixity, so there are trivial changes scattered about, and
you'll need to recompile everything.

In TysWiredIn I took the opportunity to simplify pcDataCon slightly,
by eliminating the unused Theta argument.

Do a much better job of slurping RULES.  

Now that stuff is slurped in lazily, as the simplifier pokes on it,
we may not get the rules as early as we might wish.  In the current
HEAD, no new rules are slurped in after the beginning of SimplCore,
and that means we permanently miss many rules.

This commit arranges that every time round the simplifier loop we
slurp in any new rules, and put them into the in-scope set, where the
simplifier can find them.

It's still possible that a rule might be slurped in a little later than
in earlier versions of GHC, leading to more simplifier iterations,
but let's see if that turns out to be a problem in practice.

Elaborate IfaceSyn.HsWorker to give the full IfaceExtName of the worker,
rather than just the internal OccName.  Very occasionally, the worker for
a function in module A turns out to be (by simplification) a function
defined in module B.  So we must remember the module. This shows up in
package ObjectIO,
	Graphics.UI.ObjectIO.OS.Window.osValidateWindowRect
which has a worker
	Graphics.UI.ObjectIO.OS.WindowCCall_12.$wwinValidateRect

*** Unfortunately this changes the binary format of hi files slightly, so
*** you'll have to recompile all your libraries from scratch.

Get rid of DiscardedInfo, and fix a Ghci bug at the same time.

The new story is this:

- We always read the whole interface file, as it exists on disk,
  not dropping pragmas or anything.

- We compare that from-the-disk copy with the new version before
  writing the new interface file.

- We drop the pragmas
  a) Before loading the interface payload into the declaration pools
  b) In the no-need-to-recompile case, before typechecking the
	interface decls.  Omitting this was the previous bug.

------------------------
	More newtype clearing up
	------------------------

* Change the representation of TyCons so that it accurately reflects
	* data     (0 or more constrs)
	* newtype  (1 constr)
	* abstract (unknown)
  Replaces DataConDetails and AlgTyConFlavour with AlgTyConRhs

* Add IfaceSyn.IfaceConDecls, a kind of stripped-down analogue
  of AlgTyConRhs

* Move NewOrData from BasicTypes to HsDecl (it's now an HsSyn thing)

* Arrange that Type.newTypeRep and splitRecNewType_maybe unwrap just
  one layer of new-type-ness, leaving the caller to recurse.

  This still leaves typeRep and repType in Type.lhs; these functions
  are still vaguely disturbing and probably should get some attention.

Lots of knock-on changes.  Fixes bug in ds054.

Sanity checking

Wibbles to exporting types abstractly

Trim imports

----------------------------
        Re-do kind inference (again)
	----------------------------

   [WARNING: interface file binary representation has
   (as usual) changed slightly; recompile your libraries!]

Inspired by the lambda-cube, for some time GHC has used
	type Kind = Type
That is, kinds were represented by the same data type as types.

But GHC also supports unboxed types and unboxed tuples, and these
complicate the kind system by requiring a sub-kind relationship.
Notably, an unboxed tuple is acceptable as the *result* of a
function but not as an *argument*.  So we have the following setup:

		 ?
		/ \
	       /   \
	      ??   (#)
	     /  \
            *   #

where	*    [LiftedTypeKind]   means a lifted type
	#    [UnliftedTypeKind] means an unlifted type
	(#)  [UbxTupleKind]     means unboxed tuple
	??   [ArgTypeKind]      is the lub of *,#
	?    [OpenTypeKind]	means any type at all

In particular:

  error :: forall a:?. String -> a
  (->)  :: ?? -> ? -> *
  (\(x::t) -> ...)	Here t::?? (i.e. not unboxed tuple)

All this has beome rather difficult to accommodate with Kind=Type, so this
commit splits the two.

  * Kind is a distinct type, defined in types/Kind.lhs

  * IfaceType.IfaceKind disappears: we just re-use Kind.Kind

  * TcUnify.unifyKind is a distinct unifier for kinds

  * TyCon no longer needs KindCon and SuperKindCon variants

  * TcUnify.zapExpectedType takes an expected Kind now, so that
    in TcPat.tcMonoPatBndr we can express that the bound variable
    must have an argTypeKind (??).

The big change is really that kind inference is much more systematic and
well behaved.  In particular, a kind variable can unify only with a
"simple kind", which is built from * and (->).  This deals neatly
with awkward questions about how we can combine sub-kinding with type
inference.

Lots of small consequential changes, especially to the kind-checking
plumbing in TcTyClsDecls.  (We played a bit fast and loose before, and
now we have to be more honest, in particular about how kind inference
works for type synonyms.  They can have kinds like (* -> #), so

This cures two long-standing SourceForge bugs

* 753777 (tcfail115.hs), which used erroneously to pass,
  but crashed in the code generator
      type T a = Int -> (# Int, Int #)
      f :: T a -> T a
      f t = \x -> case t x of r -> r

* 753780 (tc167.hs), which used erroneously to fail
      f :: (->) Int# Int#


Still, the result is not entirely satisfactory.  In particular

* The error message from tcfail115 is pretty obscure

* SourceForge bug 807249 (Instance match failure on openTypeKind)
  is not fixed.  Alas.

---------------------------------
  Gate in a few more instance decls
  ---------------------------------

When we have
	instance T a where...

we need to suck in the instance decl rather more aggressively than we
were.  (A case I hadn't thought of because it only happens with
undecideable instances.)

Rule selection test was exactly backwards

Add accurate source location annotations to HsSyn
-------------------------------------------------

Every syntactic entity in HsSyn is now annotated with a SrcSpan, which
details the exact beginning and end points of that entity in the
original source file.  All honest compilers should do this, and it was
about time GHC did the right thing.

The most obvious benefit is that we now have much more accurate error
messages; when running GHC inside emacs for example, the cursor will
jump to the exact location of an error, not just a line somewhere
nearby.  We haven't put a huge amount of effort into making sure all
the error messages are accurate yet, so there could be some tweaking
still needed, although the majority of messages I've seen have been
spot-on.

Error messages now contain a column number in addition to the line
number, eg.

   read001.hs:25:10: Variable not in scope: `+#'

To get the full text span info, use the new option -ferror-spans.  eg.

   read001.hs:25:10-11: Variable not in scope: `+#'

I'm not sure whether we should do this by default.  Emacs won't
understand the new error format, for one thing.

In a more elaborate editor setting (eg. Visual Studio), we can arrange
to actually highlight the subexpression containing an error.  Eventually
this information will be used so we can find elements in the abstract
syntax corresponding to text locations, for performing high-level editor
functions (eg. "tell me the type of this expression I just highlighted").

Performance of the compiler doesn't seem to be adversely affected.
Parsing is still quicker than in 6.0.1, for example.

Implementation:

This was an excrutiatingly painful change to make: both Simon P.J. and
myself have been working on it for the last three weeks or so.  The
basic changes are:

 - a new datatype SrcSpan, which represents a beginning and end position
   in a source file.

 - To reduce the pain as much as possible, we also defined:

      data Located e = L SrcSpan e

 - Every datatype in HsSyn has an equivalent Located version.  eg.

      type LHsExpr id = Located (HsExpr id)

   and pretty much everywhere we used to use HsExpr we now use
   LHsExpr.  Believe me, we thought about this long and hard, and
   all the other options were worse :-)


Additional changes/cleanups we made at the same time:

  - The abstract syntax for bindings is now less arcane.  MonoBinds
    and HsBinds with their built-in list constructors have gone away,
    replaced by HsBindGroup and HsBind (see HsSyn/HsBinds.lhs).

  - The various HsSyn type synonyms have now gone away (eg. RdrNameHsExpr,
    RenamedHsExpr, and TypecheckedHsExpr are now HsExpr RdrName,
    HsExpr Name, and HsExpr Id respectively).

  - Utilities over HsSyn are now collected in a new module HsUtils.
    More stuff still needs to be moved in here.

  - MachChar now has a real Char instead of an Int.  All GHC versions that
    can compile GHC now support 32-bit Chars, so this was a simplification.

Import all instances in cases like C (a b)

This commit does a long-overdue tidy-up

* Remove PprType (gets rid of one more bunch of hi-boot files)

* Put pretty-printing for types in TypeRep

* Make a specialised pretty-printer for Types, rather than
  converting to IfaceTypes and printing those

Improve error message when iface decl not found

When type-checking an interface in --make, when the source file hasn't
changed, we must bring into scope all the things defined in the interface.
This was breaking --make badly.

The epicentre here is TcIface.typecheckIface

Deal corectly with rules for Ids defined in this module,
even when they are imported (as orphans) from other modules.

The epicentre for this stuff is SimplCore.

Arrange that loadImportedRules can see the module dependencies
of this module, and hence know whether or not to load an hi-boot
interface.

Make rule importing work properly

Wibbles

Wibles

Oops; forgot to add this entire directory!