GitLab

Reviewed By: nomeata, thomie

Differential Revision: https://phabricator.haskell.org/D760Signed-off-by: Dave Laing <dave.laing.80@gmail.com>

Summary:
Add a reference note to each AnnKeywordId haddock comment so GHC
developers will have an idea why they are there.

Add a new test to ghc-api/annotations to serve as a template for other
GHC developers when they need to update the parser. It provides output
which checks that each SrcSpan that an annotation is attached to
actually appears in the `ParsedSource`, and lists the individual
annotations. The idea is that a developer writes a version of this
which parses a sample file using whatever syntax is changed in
Parser.y, and can then check that all the annotations come through.

Depends on D538

Test Plan: ./validate

Reviewers: simonpj, hvr, austin

Reviewed By: austin

Subscribers: thomie, jstolarek

Differential Revision: https://phabricator.haskell.org/D620

Summary:
HsTyLit now has SourceText

Update documentation of HsSyn to reflect which annotations are attached to which element.

Ensure that the parser always keeps HsSCC and HsTickPragma values, to
be ignored in the desugar phase if not needed

Bringing in SourceText for pragmas

Add Location in NPlusKPat

Add Location in FunDep

Make RecCon payload Located

Explicitly add AnnVal to RdrName where it is compound

Add Location in IPBind

Add Location to name in IEThingAbs

Add Maybe (Located id,Bool) to Match to track fun_id,infix
  This includes converting Match into a record and adding a note about why
  the fun_id needs to be replicated in the Match.

Add Location in KindedTyVar

Sort out semi-colons for parsing

  - import statements
  - stmts
  - decls
  - decls_cls
  - decls_inst

This updates the haddock submodule.

Test Plan: ./validate

Reviewers: hvr, austin, goldfire, simonpj

Reviewed By: simonpj

Subscribers: thomie, carter

Differential Revision: https://phabricator.haskell.org/D538

Summary:
As proposed in [1], this extension introduces a new syntactic form
`static e`, where `e :: a` can be any closed expression. The static form
produces a value of type `StaticPtr a`, which works as a reference that
programs can "dereference" to get the value of `e` back. References are
like `Ptr`s, except that they are stable across invocations of a
program.

The relevant wiki pages are [2, 3], which describe the motivation/ideas
and implementation plan respectively.

[1] Jeff Epstein, Andrew P. Black, and Simon Peyton-Jones. Towards
Haskell in the cloud. SIGPLAN Not., 46(12):118–129, September 2011. ISSN
0362-1340.
[2] https://ghc.haskell.org/trac/ghc/wiki/StaticPointers
[3] https://ghc.haskell.org/trac/ghc/wiki/StaticPointers/ImplementationPlanAuthored-by: Facundo Domínguez <facundo.dominguez@tweag.io>
Authored-by: Mathieu Boespflug <m@tweag.io>
Authored-by: Alexander Vershilov <alexander.vershilov@tweag.io>

Test Plan: `./validate`

Reviewers: hvr, simonmar, simonpj, austin

Reviewed By: simonpj, austin

Subscribers: qnikst, bgamari, mboes, carter, thomie, goldfire

Differential Revision: https://phabricator.haskell.org/D550

GHC Trac Issues: #7015

Summary:
Make HsLit and OverLitVal have original source strings, for source to
source conversions using the GHC API

This is part of the ongoing AST Annotations work, as captured in
https://ghc.haskell.org/trac/ghc/wiki/GhcAstAnnotations and
https://ghc.haskell.org/trac/ghc/ticket/9628#comment:28

The motivations for the literals is as follows

```lang=haskell
x,y :: Int
x = 0003
y = 0x04

s :: String
s = "\x20"

c :: Char
c = '\x20'

d :: Double
d = 0.00

blah = x
  where
    charH = '\x41'#
    intH = 0004#
    wordH = 005##
    floatH = 3.20#
    doubleH = 04.16##
    x = 1
```

Test Plan: ./sh validate

Reviewers: simonpj, austin

Reviewed By: simonpj, austin

Subscribers: thomie, goldfire, carter, simonmar

Differential Revision: https://phabricator.haskell.org/D412

GHC Trac Issues: #9628

Summary:
The final design and discussion is captured at
https://ghc.haskell.org/trac/ghc/wiki/GhcAstAnnotations

This is a proof of concept implementation of a completely
separate annotation structure, populated in the parser,and tied to the
AST by means of a virtual "node-key" comprising the surrounding
SrcSpan and a value derived from the specific constructor used for the
node.

The key parts of the design are the following.

== The Annotations ==

In `hsSyn/ApiAnnotation.hs`

```lang=haskell
type ApiAnns = (Map.Map ApiAnnKey SrcSpan, Map.Map SrcSpan [Located Token])

type ApiAnnKey = (SrcSpan,AnnKeywordId)

-- ---------------------------------------------------------------------

-- | Retrieve an annotation based on the @SrcSpan@ of the annotated AST
-- element, and the known type of the annotation.
getAnnotation :: ApiAnns -> SrcSpan -> AnnKeywordId -> Maybe SrcSpan
getAnnotation (anns,_) span ann = Map.lookup (span,ann) anns

-- |Retrieve the comments allocated to the current @SrcSpan@
getAnnotationComments :: ApiAnns -> SrcSpan -> [Located Token]
getAnnotationComments (_,anns) span =
  case Map.lookup span anns of
    Just cs -> cs
    Nothing -> []

-- | Note: in general the names of these are taken from the
-- corresponding token, unless otherwise noted
data AnnKeywordId
         = AnnAs
         | AnnBang
         | AnnClass
         | AnnClose -- ^ } or ] or ) or #) etc
         | AnnComma
         | AnnDarrow
         | AnnData
         | AnnDcolon
         ....
```

== Capturing in the lexer/parser ==

The annotations are captured in the lexer / parser by extending PState to include a field

In `parser/Lexer.x`

```lang=haskell
data PState = PState {
        ....
        annotations :: [(ApiAnnKey,SrcSpan)]
        -- Annotations giving the locations of 'noise' tokens in the
        -- source, so that users of the GHC API can do source to
        -- source conversions.
     }
```

The lexer exposes a helper function to add an annotation

```lang=haskell
addAnnotation :: SrcSpan -> Ann -> SrcSpan -> P ()
addAnnotation l a v = P $ \s -> POk s {
  annotations = ((AK l a), v) : annotations s
  } ()

```

The parser also has some helper functions of the form

```lang=haskell
type MaybeAnn = Maybe (SrcSpan -> P ())

gl = getLoc
gj x = Just (gl x)

ams :: Located a -> [MaybeAnn] -> P (Located a)
ams a@(L l _) bs = (mapM_ (\a -> a l) $ catMaybes bs) >> return a
```

This allows annotations to be captured in the parser by means of

```
ctypedoc :: { LHsType RdrName }
        : 'forall' tv_bndrs '.' ctypedoc {% hintExplicitForall (getLoc $1) >>
                                            ams (LL $ mkExplicitHsForAllTy $2 (noLoc []) $4)
                                                [mj AnnForall $1,mj AnnDot $3] }
        | context '=>' ctypedoc         {% ams (LL $ mkQualifiedHsForAllTy   $1 $3)
                                               [mj AnnDarrow $2] }
        | ipvar '::' type               {% ams (LL (HsIParamTy (unLoc $1) $3))
                                               [mj AnnDcolon $2] }
        | typedoc                       { $1 }
```

== Parse result ==

```lang-haskell
data HsParsedModule = HsParsedModule {
    hpm_module    :: Located (HsModule RdrName),
    hpm_src_files :: [FilePath],
       -- ^ extra source files (e.g. from #includes).  The lexer collects
       -- these from '# <file> <line>' pragmas, which the C preprocessor
       -- leaves behind.  These files and their timestamps are stored in
       -- the .hi file, so that we can force recompilation if any of
       -- them change (#3589)
    hpm_annotations :: ApiAnns
  }

-- | The result of successful parsing.
data ParsedModule =
  ParsedModule { pm_mod_summary   :: ModSummary
               , pm_parsed_source :: ParsedSource
               , pm_extra_src_files :: [FilePath]
               , pm_annotations :: ApiAnns }
```

This diff depends on D426

Test Plan: sh ./validate

Reviewers: austin, simonpj, Mikolaj

Reviewed By: simonpj, Mikolaj

Subscribers: Mikolaj, goldfire, thomie, carter

Differential Revision: https://phabricator.haskell.org/D438

GHC Trac Issues: #9628

This commit also refactors a bunch of lexeme-oriented code into
a new module Lexeme, and includes a submodule update for haddock.

Test Plan: Compiled ghc fine.  Opened ghci and fed it invalid code.  It gave the improved error messages in response.

Reviewers: austin

Subscribers: thomie, simonpj, spacekitteh, rwbarton, simonmar, carter

Differential Revision: https://phabricator.haskell.org/D201

Summary:
The last three '#define ...' macros were removed from Parser.y.pp and this file was renamed to Parser.y.
This basically got rid of a CPP step in the build.

Also converted two modules in compiler/parser/ from .lhs to .hs.

Test Plan: Does it build? Yes, I performed a full build here and things are looking good.

Reviewers: austin

Reviewed By: austin

Subscribers: adamse, thomie, carter, simonmar

Differential Revision: https://phabricator.haskell.org/D411

A premature complaint was causing Trac #9634.  Acutally this
change also simplifies the lexer and eliminates duplication.
(The renamer was already making the check, as it happens.)

Summary:
This includes pretty much all the changes needed to make `Applicative`
a superclass of `Monad` finally. There's mostly reshuffling in the
interests of avoid orphans and boot files, but luckily we can resolve
all of them, pretty much. The only catch was that
Alternative/MonadPlus also had to go into Prelude to avoid this.

As a result, we must update the hsc2hs and haddock submodules.
Signed-off-by: Austin Seipp <austin@well-typed.com>

Test Plan: Build things, they might not explode horribly.

Reviewers: hvr, simonmar

Subscribers: simonmar

Differential Revision: https://phabricator.haskell.org/D13

Summary:
I tried reading the lexer and the 2010 report side-by-side. Althought I didn't
quite finish, here are some small discrepancies that I found.

This revision may be low priority for reviewers, but having these commits just
in my local repository does no good either.

Changes:
* $nl was defined, but not used anywhere
* formfeed is a newline character
* add \: to $ascsymbol
  For simplification reason, the colon (':') was added to the character
  set $ascsymbol in the 2010 report. Here we make the same change.
* introduce the macros `qvarid`, `qconid`, `qvarsym` and `qconsym`
* foreign is a Haskell keyword
* add/update comments

Test Plan: Harbormaster (is awesome)

Reviewers: simonmar, hvr, austin

Reviewed By: austin

Subscribers: hvr, simonmar, ezyang, carter

Differential Revision: https://phabricator.haskell.org/D180

This also removes the short-lived NO_OVERLAP pragama, and renames
OVERLAP to OVERLAPS.

An instance may be annotated with one of 4 pragams, to control its
interaction with other overlapping instances:

  * OVERLAPPABLE:
    this instance is ignored if a more specific candidate exists

  * OVERLAPPING:
    this instance is preferred over more general candidates

  * OVERLAPS:
    both OVERLAPPING and OVERLAPPABLE (i.e., the previous GHC behavior).
    When compiling with -XOverlappingInstances, all instance are OVERLAPS.

  * INCOHERENT:
    same as before (see manual for details).
    When compiling with -XIncoherentInstances, all instances are INCOHERENT.

Summary:
Previously, both Cabal and GHC defined the type PackageId, and we expected
them to be roughly equivalent (but represented differently).  This refactoring
separates these two notions.

A package ID is a user-visible identifier; it's the thing you write in a
Cabal file, e.g. containers-0.9.  The components of this ID are semantically
meaningful, and decompose into a package name and a package vrsion.

A package key is an opaque identifier used by GHC to generate linking symbols.
Presently, it just consists of a package name and a package version, but
pursuant to #9265 we are planning to extend it to record other information.
Within a single executable, it uniquely identifies a package.  It is *not* an
InstalledPackageId, as the choice of a package key affects the ABI of a package
(whereas an InstalledPackageId is computed after compilation.)  Cabal computes
a package key for the package and passes it to GHC using -package-name (now
*extremely* misnamed).

As an added bonus, we don't have to worry about shadowing anymore.

As a follow on, we should introduce -current-package-key having the same role as
-package-name, and deprecate the old flag.  This commit is just renaming.

The haddock submodule needed to be updated.
Signed-off-by: Edward Z. Yang <ezyang@cs.stanford.edu>

Test Plan: validate

Reviewers: simonpj, simonmar, hvr, austin

Subscribers: simonmar, relrod, carter

Differential Revision: https://phabricator.haskell.org/D79

Conflicts:
	compiler/main/HscTypes.lhs
	compiler/main/Packages.lhs
	utils/haddock

Programmers may provide a pragma immediately after the `instance` keyword
to control the overlap/incoherence behavior for individual instances.
For example:

    instance {-# OVERLAP #-} C a where ...

I chose this notation, rather than the other two outlined in the ticket
for these reasons:

   1. Having the pragma after the type looks odd, I think.
   2. Having the pragma after there `where` does not work for
       stand-alone derived instances

I have implemented 3 pragams:

   1. NO_OVERLAP
   2. OVERLAP
   3. INCOHERENT

These correspond directly to the internal modes currently supported by
GHC.  If a pragma is specified, it will be used no matter what flags are
turned on.   For example, putting `NO_OVERLAP` on an instance will mark
it as non-overlapping, even if `OVERLAPPIN_INSTANCES` is turned on for the
module.

Haskell2010 supports

- base-10 (prefix-less),
- base-8 (via `0[oO]`-prefix), and
- base-16 (via `0[xX]`-prefix) integer literals.

This commit adds syntax support for base-2 integer literals via the new `0[bB]`
prefix. The use of a `0b` prefix for indicating binary literals is known
from popular programming languages such as C++14, Perl, Python, Ruby, and Java.

This syntax extension is disabled by default and can be enabled via the
new `{-# LANGUAGE BinaryLiterals #-}` pragma and/or the new `-XBinaryLiterals`

This new extensions requires to upgrade the `ExtsBitmap` type from
`Word` to `Word64` as this adds a 33th flag which is not guaranteed to
fit into a `Word`.
Signed-off-by: Herbert Valerio Riedel <hvr@gnu.org>

Differential Revision: https://phabricator.haskell.org/D22

This introduces an Enum type `ExtBits` for the currently used 32 flags
and introduces a type-synonym `ExtsBitmap` for representing a set over
`ExtBits`. While at it, the current `Int` was replaced by `Word` to
have the compiler catch any missed use-sites.

This will make it easy to swap the `Word`-representation of `ExtsBitmap`
by something different, such as e.g. a `Word64`

Test Plan: successful validate

Differential Revision: https://phabricator.haskell.org/D23

This patch implements Pattern Synonyms (enabled by -XPatternSynonyms),
allowing y ou to assign names to a pattern and abstract over it.

The rundown is this:

  * Named patterns are introduced by the new 'pattern' keyword, and can
    be either *unidirectional* or *bidirectional*. A unidirectional
    pattern is, in the simplest sense, simply an 'alias' for a pattern,
    where the LHS may mention variables to occur in the RHS. A
    bidirectional pattern synonym occurs when a pattern may also be used
    in expression context.

  * Unidirectional patterns are declared like thus:

        pattern P x <- x:_

    The synonym 'P' may only occur in a pattern context:

        foo :: [Int] -> Maybe Int
        foo (P x) = Just x
        foo _     = Nothing

  * Bidirectional patterns are declared like thus:

        pattern P x y = [x, y]

    Here, P may not only occur as a pattern, but also as an expression
    when given values for 'x' and 'y', i.e.

        bar :: Int -> [Int]
        bar x = P x 10

  * Patterns can't yet have their own type signatures; signatures are inferred.

  * Pattern synonyms may not be recursive, c.f. type synonyms.

  * Pattern synonyms are also exported/imported using the 'pattern'
    keyword in an import/export decl, i.e.

        module Foo (pattern Bar) where ...

    Note that pattern synonyms share the namespace of constructors, so
    this disambiguation is required as a there may also be a 'Bar'
    type in scope as well as the 'Bar' pattern.

  * The semantics of a pattern synonym differ slightly from a typical
    pattern: when using a synonym, the pattern itself is matched,
    followed by all the arguments. This means that the strictness
    differs slightly:

        pattern P x y <- [x, y]

        f (P True True) = True
        f _             = False

        g [True, True] = True
        g _            = False

    In the example, while `g (False:undefined)` evaluates to False,
    `f (False:undefined)` results in undefined as both `x` and `y`
    arguments are matched to `True`.

For more information, see the wiki:

    https://ghc.haskell.org/trac/ghc/wiki/PatternSynonyms
    https://ghc.haskell.org/trac/ghc/wiki/PatternSynonyms/ImplementationReviewed-by: Simon Peyton Jones <simonpj@microsoft.com>
Signed-off-by: Austin Seipp <austin@well-typed.com>

The lexer now uses unicode single quotation marks in its error messages
if possible.  This is due to the use of the 'quotes' combinator.

Patch by errge

Right now the syntax for typed expression brackets and splices maps to
conventional brackets and splices, i.e., they are not typed.

This makes it possible to write

x = if | False -> if | False -> 1
                     | False -> 2
       | True -> 3

Layout normally inserts semicolons between declarations at the same
indentation level, so I added optional semicolons to the syntax for
guards in MultiWayIf syntax.  This is a bit of a hack, but the
alternative (a special kind of layout that doesn't insert semicolons)
seemed worse, or at least equally bad.

This commit adds a `{-# MINIMAL #-}` pragma, which defines the possible
minimal complete definitions for a class. The body of the pragma is a
boolean formula of names.

The old warning for missing methods is replaced with this new one.

Note: The interface file format is changed to store the minimal complete
definition.
Authored-by: Twan van Laarhoven <twanvl@gmail.com>
Signed-off-by: Herbert Valerio Riedel <hvr@gnu.org>

This fixes bugs #8185, #8234, and #8246. The new syntax is explained
in the comments to #8185, appears in the "Roles" subsection of the
manual, and on the [wiki:Roles] wiki page.

This change also removes the ability for a role annotation on type
synonyms, as noted in #8234.

This patch encompasses most of the basic infrastructure for GHCJS. It
includes:

  * A new extension, -XJavaScriptFFI
  * A new architecture, ArchJavaScript
  * Parser and lexer support for 'foreign import javascript', only
    available under -XJavaScriptFFI, using ArchJavaScript.
  * As a knock-on, there is also a new 'WayCustom' constructor in
    DynFlags, so clients of the GHC API can add custom 'tags' to their
    built files. This should be useful for other users as well.

The remaining changes are really just the resulting fallout, making sure
all the cases are handled appropriately for DynFlags and Platform.
Authored-by: Luite Stegeman <stegeman@gmail.com>
Signed-off-by: Austin Seipp <aseipp@pobox.com>

Fixes Trac #7901.

'∀' is neither upper nor lowercase, unlike the 'f' in 'forall', so when
explicit forall is not enabled, it creates a parse error before reaching
the '.', which is where we give a nice message for ascii 'forall'.
Therefore, we make '∀' into a token as long as UnicodeSyntax is enabled,
which is safe because its caselessness means it can never be mistaken
for a symbol, and check extensions in the parser when the 'forall' rule
is used.
Authored-by: Paul Cavallaro <ptc@fb.com>
Authored-by: Anders Papitto <anderspapitto@gmail.com>
Signed-off-by: Austin Seipp <aseipp@pobox.com>

Signed-off-by: Austin Seipp <aseipp@pobox.com>

Roles are a solution to the GeneralizedNewtypeDeriving type-safety
problem.

Roles were first described in the "Generative type abstraction" paper,
by Stephanie Weirich, Dimitrios Vytiniotis, Simon PJ, and Steve Zdancewic.
The implementation is a little different than that paper. For a quick
primer, check out Note [Roles] in Coercion. Also see
http://ghc.haskell.org/trac/ghc/wiki/Roles
and
http://ghc.haskell.org/trac/ghc/wiki/RolesImplementation
For a more formal treatment, check out docs/core-spec/core-spec.pdf.

This fixes Trac #1496, #4846, #7148.

I'd been meaning to do this for some time, but finally got around to it
due to the overflowing literals warning. With that enabled, we were
getting a warning for
    -128 :: Int8
as that is parsed as
    negate (fromInteger 128)
which just happens to do the right thing, as
    negate (fromInteger 128) = negate (-128) = -128

(# is now always a lexeme, even if followed by a symbol.

Mostly d -> g (matching DynFlag -> GeneralFlag).
Also renamed if* to when*, matching the Haskell if/when names