This patch implements three significant improvements to Template Haskell.
  
Declaration-level splices with no "$" 
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
This change simply allows you to omit the "$(...)" wrapper for 
declaration-level TH splices.  An expression all by itself is
not legal, so we now treat it as a TH splice.  Thus you can now
say
	data T = T1 | T2
 	deriveMyStuff ''T

where deriveMyStuff :: Name -> Q [Dec]
This makes a much nicer interface for clients of libraries that use
TH: no scary $(deriveMyStuff ''T).

Nested top-level splices
~~~~~~~~~~~~~~~~~~~~~~~~
Previously TH would reject this, saying that splices cannot be nested:
	f x = $(g $(h 'x))
But there is no reason for this not to work.  First $(h 'x) is run,
yielding code <blah> that is spliced instead of the $(h 'x). Then (g
<blah>) is typechecked and run, yielding code that replaces the
$(g ...) splice.  

So this simply lifts the restriction.

Fix Trac #3467: non-top-level type splices
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

It appears that when I added the ability to splice types in TH
programs, I failed to pay attention to non-top-level splices -- that
is, splices inside quotatation brackets.  

This patch fixes the problem.  I had to modify HsType, so there's a
knock-on change to Haddock.

Its seems that a lot of lines of code has changed, but almost all the
new lines are comments!

General tidying up
~~~~~~~~~~~~~~~~~~
As a result of thinking all this out I re-jigged the data type ThStage,
which had far too many values before.  And I wrote a nice state transition
diagram to make it all precise; 
   see Note [Template Haskell state diagram] in TcSplice

Lots more refactoring in TcSplice, resulting in significantly less code.
(A few more lines, but actually less code -- the rest is comments.)

I think the result is significantly cleaner.

This is a minor change to the parser that tidies it up a bit,
and allows us to parse

	data T :: *
        data S :: * -> *

just like  

	data T 
	data S a

 - the package DB is a directory containing one file per package
   instance (#723)

 - there is a binary cache of the database (#593, #2089)

 - the binary package is now a boot package

 - there is a new package, bin-package-db, containing the Binary
   instance of InstalledPackageInfo for the binary cache.

Also included in this patch

 - Use colour in 'ghc-pkg list' to indicate broken or hidden packages
  
   Broken packages are red, hidden packages are 
  
   Colour support comes from the terminfo package, and is only used when
    - not --simple-output
    - stdout is a TTY
    - the terminal type has colour capability

 - Fix the bug that 'ghc-pkg list --user' shows everything as broken

Make it clear that Haskell code to be used by other Haskell code
must be built as a package.

Now, adding dyn to $(GhcLibWays) is how shared libs are enabled.
Everything else keys off that, rather than testing
$(BuildSharedLibs).

A few changes bundled together here:

 - Add -package-id flag, like -package but takes an
   InstalledPackageId as an argument (mainly for Cabal)
   (docs to follow)

 - Fix the shadowing behaviour: if we end up with
   two packages with the same name/version that came from
   different package databases, then one will shadow the
   other.  This may mean we have to disable other packages
   that depended on the now-shadowed package.  Lots of
   refactoring to ensure that we can get reasonable diagnostics when
   this happens

<command line>: cannot satisfy -package shadowdep: 
    shadowdep-1-XXX is unusable due to missing or recursive dependencies:
      shadow-1-XXX
    (use -v for more information)

and fix up related cruft

A fine bug report (#3403) demonstrated that we were losing the tail
call property when a complicated pattern match was involved.  After
a bit of investigation I discovered that the culprit was the failure
join-point introduced by the pattern matcher.  It was a zero-argument
thunk, which is not very CPR-friendly, and that interacted badly with
CPR worker/wrapper.

It's easy to fix, the same way that we fix other join points, by supplying
a dummy argument (that is not really passed at runtime.

When checking the interface exported by a hs-boot file against the
Real Thing, I'd failed to check the kind of a type constructor.  If you
get it wrong, the inconsistency leads to all manner of mischief, as 
'wkahl' reports in #3468.

This patch should do the job.

  - tracing facilities are now enabled with -DTRACING, and -DDEBUG
    additionally enables debug-tracing.  -DEVENTLOG has been
    removed.

  - -debug now implies -eventlog

  - events can be printed to stderr instead of being sent to the
    binary .eventlog file by adding +RTS -v (which is implied by the
    +RTS -Dx options).

  - -Dx debug messages can be sent to the binary .eventlog file
    by adding +RTS -l.  This should help debugging by reducing
    the impact of debug tracing on execution time.

  - Various debug messages that duplicated the information in events
    have been removed.