The GHC Commentary

This tree of wiki pages is a "commentary" on the GHC source code. It contains all the explanatory material that doesn't belong in comments in the source code itself, because the material is wide-ranging, usually covers multiple source files, and is more architectural in nature. The commentary can also be considered a design document for GHC.

For the beginners there is a short getting started guide.

For the dedicated, there are videos of Simon and Simon giving an overview of GHC, at the 2006 GHC Hackathon.

Also check out the GHC Reading List, which gives lots of background reading that will help you understand the actual implementation. Here's another reading list from Stephen Diehl.

Stephen also has a helpful series of blog posts about GHC internals

• Dive into GHC: pipeline
• Dive into GHC: intermediate forms
• Dive into GHC: targeting Core

Editing the Commentary

Please feel free to add material to the rest of the wiki: don't worry too much about accuracy (in due course someone will edit your contribution). When unsure though please indicate this; it's best to ask on the GHC mailing list so you can correct the commentary. Please give some thought to where in the commentary your contribution belongs. GHC has an older commentary (non wiki based) that read like a single coherent narrative, made sure to define terms before using them, and introduced concepts in the order which made them easiest to understand. Please do try to preserve those properties in this wiki commentary. If you're unsure or in a hurry, consider creating a wiki page outside the commentary and linking to it from the commentary (or the "contributed documentation" section below).

Also try to add appropriate links to other parts of the commentary.

Contents

• Getting Started

  • Source Tree Roadmap
  • Module Structure
  • Documentation Style
  • Coding Style
  • Abbreviations in GHC
  • Platforms and their Naming Convention

• The Compiler

• The Libraries on which GHC depends

  • The Integer libraries (`integer-gmp` and `integer-simple`)

• The Runtime System (RTS)

  • RTS Coding Conventions
  • The Haskell Execution Model
  • The memory layout of heap and stack objects

• Cross-cutting concerns: topics which span both the compiler and the runtime system

  • Profiling
  • Wired-in and known-key things
  • Primitive Operations (PrimOps)
  • The Package System

• The User Manual (formatting guidelines etc)

Contributed Documentation

The above commentary covers the source code of GHC. For material that doesn't concern this topic (such as proposals, work-in-progress and status reports) or that don't fit into the existing structure, you will find them below. Feel free to add new material here but please categorise it correctly.

• General Notes on the GHC compiler

  • Ningning Xie's literature review of various contributions to GHC's type system

  • Edward Yang's blog post about the entire compilation pipeline for `factorial`

  • New Prim Ops: How to add new primitive operations to GHC Haskell.

  • Replacing GMP: Notes from an effort to replace GMP with another Bignum library.

  • External Core: Describes the process of bringing External Core up to speed. Once finished, this will simply describe what External Core is, and how it works.

  • The Scrap your boilerplate homepage.

  • Optimisation Ordering Describe the ordering and interaction of optimisation passes (Old).

  • GHC Illustrated (follow the PDF link), a very insightful tutorial on GHC's internals.

  • Ollie Charles's 24 days of GHC Extensions, and Lennart Augstsson's commentary

    • Welcome
    • Static Pointers
    • Template Haskell
    • Arrows
    • Scoped Type Variables
    • Existential Quantification
    • Rank N Types
    • Overloaded Strings
    • DeriveGeneric
    • Deriving
    • Functional Dependencies
    • Multi-parameter Type Classes
    • Type Families
    • Implicit Parameters
    • Nullary Type Classes
    • Recursive Do
    • Type Operators
    • List Comprehensions
    • Rebindable Syntax
    • Bang Patterns
    • Record Wildcards
    • Pattern Synonyms
    • View Patterns
    • Thanks

  • Commentary/Rts/CompilerWays: Compiler ways in GHC, what, how, and where

• Notes on implemented GHC features:

  • Evaluation order and state tokens: notes written by Michael Snoyberg in response to #9390 (closed).
  • Notes on fusion (eg foldr/build)
  • Overloaded list syntax allows you to use list notation for things other than lists.
  • Kind polymorphism and data type promotion
  • A kind for class constraints. Implemented as ConstraintKinds
  • LLVM back end
  • Support for generic programming
  • Notes about Template Haskell
  • Rewrite Rules: Notes about the implementation of RULEs in GHC
  • Monad Comprehensions: Translation rules and some implementation details
  • Haddock: Some notes about how the Haddock comment support is implemented.
  • Intermediate Types: Notes about the type system of GHC's new intermediate language (in the HEAD since ICFP'06)
  • Type families/type functions: Notes concerning the implementation of type families, associated types, and equality constraints as well as the extension of the type checker with a contraint solver for equality constraints.
  • Magic to do with `seq` and friends
  • Compiler plug-ins
  • memcpy/memmove/memset optimizations
  • Backend Ideas: Some ideas and notes about the back end.
  • Notes about the new code generator
  • A record of improvements made to the performance of the Hoopl library for dataflow optimisation
  • DPH: Notes about the implementation of Data Parallel Haskell
  • Safe Haskell: The design of the GHC Safe Haskell extension
  • SQL-Like Comprehensions: Notes on SPJs "Comprehensive Comprehensions" (TransformComprehensions)
  • Deferring compilation type errors to runtime (`-fdefer-type-errors`)
  • Demand analyser Notes on the meanings, worker-wrapper splitting of demand signatures and relevant components of the compiler
  • Closed type families
  • OneShot The magic oneShot function.
  • Deriving Functor, Foldable, and Traversable

• Notes on proposed or in progress (but out of tree) GHC compiler features:

  • Making Haskell strict
  • Improving pattern-match overlap and exhaustiveness checks
  • Source-locations on HsSyn
  • How GHC inter-operates with Cabal and Backpack
  • StaticValues and ticket #7015 (closed)
  • Partial type signatures and its ticket #9478 (closed)
  • Late lambda-lifting, and its ticket #9476 (closed)
  • Roles in Haskell
  • Dependent types in Haskell
  • Nested CPR analysis
  • Giving Template Haskell full access to annotations
  • Checking consistency of functional dependencies
  • Allowing multiple instances of the same package to be installed, each instance having different dependencies
  • Contracts in Haskell
  • Agda-style holes in terms which supports writing partial programs.
  • Records
  • Cloud Haskell
  • A modular package language for Haskell Scott Kilpatrick and Derek Dreyer are designing a new “package language” for Haskell in the style of the ML module system.
  • Pattern synonyms
  • Type level naturals and type level reasoning.
  • The solve for type-level naturals
  • Polymorphic Dynamic: Notes on adding ad-hoc polymorphic dynamic types
  • Proposal to allow classes to give default implementations for their superclasses
  • Cmm: Implementing Exception Handling: Implementing exception handling for primitive operations in Cmm
  • Cmm: Using Kinds to implement calling conventions: Pointers to explanations of what a CmmKind is and why you might find one useful.
  • SIMD: Notes on adding SIMD instructions and primOps to GHC
  • Explicit Call Stack: Notes about maintaining an explicit call stack, to support error attribution and profiling.
  • Objective-C FFI: Haskell FFI support for Objective-C
  • C Blocks: Haskell FFI support for block objects (closures) in C
  • Syntax for explicit type application and scoped type variables
  • Syntax for defining kinds that do not arise from datatype promotion
  • Lambda-Case: Syntax for full (with branching) pattern matching on arguments of lambda abstractions
  • Typeable: Making Typeable more expressive
  • SafeRoles: Roles & Safe Haskell

• Library issues

  • Splitting up the base package

• GHCi Debugger

  • Commentary/GHCi: Notes on the implementation details for the support of breakpoints in GHCi.
  • GHCi Debugger: Notes on the implementation details of the GHCi debugger's feature for inspecting values.

• The Runtime System

  • A new lightweight concurrency substrate for GHC
  • Garbage Collector: Notes about GHC's existing single threaded garbage collector and development of a parallel GC.
  • GMP Memory Managment: Describes how the garbage collector cooperates with GMP for Integer.
  • SemiTagging: Describes how the semi-tagging optimisation will be implemented.
  • PAPI: Measurement of program performance using CPU events (cache misses, branch mis-predictions).

• Cross-cutting concerns: topics which span both the compiler and the runtime system

  • Haskell Program Coverage: How HPC works

• Parallel Haskell Variants: All aspects of the GpH and Eden parallel variants of GHC.

Old Documentation

Here are some useful, but somewhat-out-of-date resources:

• The old GHC Commentary: Information on the internals of GHC, in various states of up-to-dateness. We are keen to move this stuff out of its current location and onto this Wiki. If anyone is willing to help do that, even for just a part in which you are interested, we would be delighted. There is a page which tracks the progress of migrating information from the old commentary to this wiki.
• GHC Papers: Papers and pointers to other documents that relate to the inner workings of GHC.