Some of the problems highlighted in trac #3968.

Inspired by Trac #3703

Trac #3734 suggested addding some extra guidance about 
incoherence and overlap; now done

Do not merge to 6.12

Merge to 6.12

Merge to 6.12 along with the main DoRec patch

The change is this (see Trac #2798).  Instead of writing

  mdo { a <- getChar
      ; b <- f c
      ; c <- g b
      ; putChar c
      ; return b }

you would write

  do { a <- getChar
     ; rec { b <- f c
           ; c <- g b }
     ; putChar c
     ; return b }

That is, 
  * 'mdo' is eliminated 
  * 'rec' is added, which groups a bunch of statements
    into a single recursive statement

This 'rec' thing is already present for the arrow notation, so it  
makes the two more uniform.  Moreover, 'rec' lets you say more
precisely where the recursion is (if you want to), whereas 'mdo' just
says "there's recursion here somewhere".  Lastly, all this works with
rebindable syntax (which mdo does not).

Currently 'mdo' is enabled by -XRecursiveDo.  So we now deprecate this
flag, with another flag -XDoRec to enable the 'rec' keyword.

Implementation notes:
  * Some changes in Lexer.x
  * All uses of RecStmt now use record syntax

I'm still not really happy with the "rec_ids" and "later_ids" in the
RecStmt constructor, but I don't dare change it without consulting Ross
about the consequences for arrow syntax.

This implements the Haskell Prime proposal
  http://hackage.haskell.org/trac/haskell-prime/wiki/ExplicitForall

Flag is -XExplicitForAll

Implied by Opt_RankNTypes, Opt_Rank2Types, Opt_ScopedTypeVariables, 
           Opt_LiberalTypeSynonyms, Opt_ExistentialQuantification, 
           Opt_PolymorphicComponents 

This patch implements three significant improvements to Template Haskell.
  
Declaration-level splices with no "$" 
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
This change simply allows you to omit the "$(...)" wrapper for 
declaration-level TH splices.  An expression all by itself is
not legal, so we now treat it as a TH splice.  Thus you can now
say
	data T = T1 | T2
 	deriveMyStuff ''T

where deriveMyStuff :: Name -> Q [Dec]
This makes a much nicer interface for clients of libraries that use
TH: no scary $(deriveMyStuff ''T).

Nested top-level splices
~~~~~~~~~~~~~~~~~~~~~~~~
Previously TH would reject this, saying that splices cannot be nested:
	f x = $(g $(h 'x))
But there is no reason for this not to work.  First $(h 'x) is run,
yielding code <blah> that is spliced instead of the $(h 'x). Then (g
<blah>) is typechecked and run, yielding code that replaces the
$(g ...) splice.  

So this simply lifts the restriction.

Fix Trac #3467: non-top-level type splices
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

It appears that when I added the ability to splice types in TH
programs, I failed to pay attention to non-top-level splices -- that
is, splices inside quotatation brackets.  

This patch fixes the problem.  I had to modify HsType, so there's a
knock-on change to Haddock.

Its seems that a lot of lines of code has changed, but almost all the
new lines are comments!

General tidying up
~~~~~~~~~~~~~~~~~~
As a result of thinking all this out I re-jigged the data type ThStage,
which had far too many values before.  And I wrote a nice state transition
diagram to make it all precise; 
   see Note [Template Haskell state diagram] in TcSplice

Lots more refactoring in TcSplice, resulting in significantly less code.
(A few more lines, but actually less code -- the rest is comments.)

I think the result is significantly cleaner.

* Make C { A.a } work with punning, expanding to C { A.a = a }

* Make it so that, with -fwarn-unused-matches, 
        f (C {..}) = x
  does not complain about the bindings introduced by the "..".

* Make -XRecordWildCards implies -XDisambiguateRecordFields.

* Overall refactoring of RnPat, which had become very crufty. 
  In particular, there is now a monad, CpsRn, private to RnPat,
  which deals with the cps-style plumbing.  This is why so many
  lines of RnPat have changed.

* Refactor the treatment of renaming of record fields into two passes
	- rnHsRecFields1, used both for patterns and expressions,
	     which expands puns, wild-cards
  	- a local renamer in RnPat for fields in patterns
	- a local renamer in RnExpr for fields in construction and update
  This make it all MUCH easier to understand
 
* Improve documentation of record puns, wildcards, and disambiguation

This patch adds tuple sections, so that

	(x,,z)  means   \y -> (x,y,z)

Thanks for Max Bolinbroke for doing the hard work.

In the end, instead of using two constructors in HsSyn, I used
just one (still called ExplicitTuple) whose arguments can be
	Present (LHsExpr id)
or	Missing PostTcType

While I was at it, I did a bit of refactoring too.

The main purpose of this patch is to fix Trac #3306, by fleshing out the
syntax for GADT-style record declraations so that you have a context in 
the type.  The new form is
   data T a where
     MkT :: forall a. Eq a => { x,y :: !a } -> T a
See discussion on the Trac ticket.

The old form is still allowed, but give a deprecation warning.

When we remove the old form we'll also get rid of the one reduce/reduce
error in the grammar. Hurrah!

While I was at it, I failed as usual to resist the temptation to do lots of
refactoring.  The parsing of data/type declarations is now much simpler and
more uniform.  Less code, less chance of errors, and more functionality.
Took longer than I planned, though.

ConDecl has record syntax, but it was not being used consistently, so I
pushed that through the compiler.

Makes GADT syntax consistent by allowing multiple constructors
to be given a single signature
   data T wehre
       A, B :: T
       C :: Int -> t

See Trac #2953. This patch implements a distinct flag for each extended
class that may be automatically derived.  And I updated the user manual
to reflect the fact that we can now derive Functor, Foldable, Traversable.