Commit 13ee9668 authored by Takenobu Tani's avatar Takenobu Tani

Update links for source code

This fix using `sed -i -e 's#/trac/ghc/browser/ghc/#https://gitlab.haskell.org/ghc/ghc/blob/master/#g'`
parent 2e13e628
......@@ -67,7 +67,7 @@ dataExecResult=ExecComplete{ execResult ::EitherSomeException[Name], execAllocat
```
Normally what happens is that `execStmt` forks a new thread to handle the evaluation of the expression. It calls `evalStmt` ([compiler/ghci/GHCi.hs](https://gitlab.haskell.org/ghc/ghc/tree/master/ghc/compiler/ghci/GHCi.hs) in both remote and normal mode) to create an `EvalStmt``Message`. This message is processed by the `evalStmt` ([libraries/ghci/GHCi/Run.hs](/trac/ghc/browser/ghc/libraries/ghci/GHCi/Run.hs) in normal mode). This in turns calls the `sandboxIO` to do `forkIO`. It then blocks on an `MVar` and waits for the thread to finish.
Normally what happens is that `execStmt` forks a new thread to handle the evaluation of the expression. It calls `evalStmt` ([compiler/ghci/GHCi.hs](https://gitlab.haskell.org/ghc/ghc/tree/master/ghc/compiler/ghci/GHCi.hs) in both remote and normal mode) to create an `EvalStmt``Message`. This message is processed by the `evalStmt` ([libraries/ghci/GHCi/Run.hs](https://gitlab.haskell.org/ghc/ghc/blob/master/libraries/ghci/GHCi/Run.hs) in normal mode). This in turns calls the `sandboxIO` to do `forkIO`. It then blocks on an `MVar` and waits for the thread to finish.
This `MVar` is (now) called `statusMVar`, because it carries the execution status of the computation which is being evaluated. We will discuss its type shortly. When the thread finishes it fills in `statusMVar`, which wakes up `execStmt`, and it returns a `ExecResult`.
......
......@@ -75,13 +75,13 @@ Here is a block diagram of its top-level structure:
The part called [HscMain](commentary/compiler/hsc-main) deals with compiling a single module. On top of this is built the **compilation manager** (in blue) that manages the compilation of multiple modules. It exports an interface called the **GHC API**. On top of this API are four small front ends:
- GHCi, the interactive environment, is implemented in [ghc/GHCi/UI.hs](/trac/ghc/browser/ghc/ghc/GHCi/UI.hs) and [compiler/main/InteractiveEval.hs](/trac/ghc/browser/ghc/compiler/main/InteractiveEval.hs). It sits squarely on top of the GHC API.
- GHCi, the interactive environment, is implemented in [ghc/GHCi/UI.hs](https://gitlab.haskell.org/ghc/ghc/blob/master/ghc/GHCi/UI.hs) and [compiler/main/InteractiveEval.hs](https://gitlab.haskell.org/ghc/ghc/blob/master/compiler/main/InteractiveEval.hs). It sits squarely on top of the GHC API.
- `--make` is almost a trivial client of the GHC API, and is implemented in [compiler/main/GhcMake.hs](/trac/ghc/browser/ghc/compiler/main/GhcMake.hs).
- `--make` is almost a trivial client of the GHC API, and is implemented in [compiler/main/GhcMake.hs](https://gitlab.haskell.org/ghc/ghc/blob/master/compiler/main/GhcMake.hs).
- `-M`, the Makefile dependency generator, is also a client of the GHC API and is implemented in [compiler/main/DriverMkDepend.hs](https://gitlab.haskell.org/ghc/ghc/tree/master/ghc/compiler/main/DriverMkDepend.hs).
......
......@@ -8,27 +8,27 @@ The register allocator code is split into two main sections, the register alloca
### The register allocator
- [compiler/nativeGen/RegLiveness.hs](/trac/ghc/browser/ghc/compiler/nativeGen/RegLiveness.hs)
- [compiler/nativeGen/RegLiveness.hs](https://gitlab.haskell.org/ghc/ghc/blob/master/compiler/nativeGen/RegLiveness.hs)
Defines `LiveInstr` and `LiveCmmTop` which carry native machine instructions annotated with register liveness information. It also provides functions to annotate native code (`NatCmmTop`) with this liveness information, and to slurp out sets of register conflicts for feeding into the coloring allocator.
- [compiler/nativeGen/RegAllocColor.hs](/trac/ghc/browser/ghc/compiler/nativeGen/RegAllocColor.hs)
- [compiler/nativeGen/RegAllocColor.hs](https://gitlab.haskell.org/ghc/ghc/blob/master/compiler/nativeGen/RegAllocColor.hs)
Defines `regAlloc`, the main driver function for the graph coloring allocator. The driver accepts `LiveCmmTop`s which use virtual regs, and produces`NatCmmTops` which use real machine regs. This module also provides functions to help build and deep seq the register conflict graph.
- [compiler/nativeGen/RegAllocLinear.hs](/trac/ghc/browser/ghc/compiler/nativeGen/RegAllocLinear.hs)
- [compiler/nativeGen/RegAllocLinear.hs](https://gitlab.haskell.org/ghc/ghc/blob/master/compiler/nativeGen/RegAllocLinear.hs)
Defines the linear scan allocator. Its interface is identical to the coloring allocator.
- [compiler/nativeGen/RegAllocInfo.hs](/trac/ghc/browser/ghc/compiler/nativeGen/RegAllocInfo.hs)
- [compiler/nativeGen/RegAllocInfo.hs](https://gitlab.haskell.org/ghc/ghc/blob/master/compiler/nativeGen/RegAllocInfo.hs)
Defines the register information function, `regUsage`, which takes a set of real and virtual registers and returns the actual registers used by a particular `Instr`; register allocation is in AT&T syntax order (source, destination), in an internal function, `usage`; defines the `RegUsage` data type
- [compiler/nativeGen/RegSpillCost.hs](/trac/ghc/browser/ghc/compiler/nativeGen/RegSpillCost.hs)
- [compiler/nativeGen/RegSpillCost.hs](https://gitlab.haskell.org/ghc/ghc/blob/master/compiler/nativeGen/RegSpillCost.hs)
Defines `chooseSpill` which is responsible for selecting a virtual reg to spill to the stack when not enough real regs are available.
- [compiler/nativeGen/RegSpill.hs](/trac/ghc/browser/ghc/compiler/nativeGen/RegSpill.hs)
- [compiler/nativeGen/RegSpill.hs](https://gitlab.haskell.org/ghc/ghc/blob/master/compiler/nativeGen/RegSpill.hs)
Defines `regSpill` which takes `LiveCmmTop`s and inserts spill/reload instructions virtual regs that wouldn't fit in real regs. `regSpill`'s strategy is to simply inserts spill/reloads for every use/def of a particular virtual reg. This inefficient code is cleaned up by the spill cleaner after allocation.
......@@ -36,44 +36,44 @@ The register allocator code is split into two main sections, the register alloca
- [compiler/nativeGen/RegSpillClean.hs](/trac/ghc/browser/ghc/compiler/nativeGen/RegSpillClean.hs)
- [compiler/nativeGen/RegSpillClean.hs](https://gitlab.haskell.org/ghc/ghc/blob/master/compiler/nativeGen/RegSpillClean.hs)
The spill cleaner is run after real regs have been allocated. It erases spill/reload instructions inserted by `regSpill` that weren't strictly nessesary.
- [compiler/nativeGen/RegAllocStats.hs](/trac/ghc/browser/ghc/compiler/nativeGen/RegAllocStats.hs)
- [compiler/nativeGen/RegAllocStats.hs](https://gitlab.haskell.org/ghc/ghc/blob/master/compiler/nativeGen/RegAllocStats.hs)
Defines data types and pretty printers used for collecting statistics and debugging info from the coloring allocator.
### Graph coloring
- [compiler/utils/GraphBase.hs](/trac/ghc/browser/ghc/compiler/utils/GraphBase.hs)
- [compiler/utils/GraphBase.hs](https://gitlab.haskell.org/ghc/ghc/blob/master/compiler/utils/GraphBase.hs)
Defines the basic `Graph`, `Node` and `Triv` types used by the coloring algorithm.
- [compiler/utils/GraphColor.hs](/trac/ghc/browser/ghc/compiler/utils/GraphColor.hs)
- [compiler/utils/GraphColor.hs](https://gitlab.haskell.org/ghc/ghc/blob/master/compiler/utils/GraphColor.hs)
Defines the function `colorGraph` which is responsible for assigning colors (real regs) to nodes (virtual regs) in the register conflict graph.
- [compiler/utils/GraphOps.hs](/trac/ghc/browser/ghc/compiler/utils/GraphOps.hs)
- [compiler/utils/GraphOps.hs](https://gitlab.haskell.org/ghc/ghc/blob/master/compiler/utils/GraphOps.hs)
Defines functions to perform basic operations on the graphs such as adding, deleting, and coalescing nodes.
- [compiler/utils/GraphPps.hs](/trac/ghc/browser/ghc/compiler/utils/GraphPps.hs)
- [compiler/utils/GraphPps.hs](https://gitlab.haskell.org/ghc/ghc/blob/master/compiler/utils/GraphPps.hs)
Defines functions for pretty print graphs in human readable-ish and graphviz format.
### Miscellanea
- [compiler/nativeGen/RegCoalesce.hs](/trac/ghc/browser/ghc/compiler/nativeGen/RegCoalesce.hs)
- [compiler/nativeGen/RegCoalesce.hs](https://gitlab.haskell.org/ghc/ghc/blob/master/compiler/nativeGen/RegCoalesce.hs)
Defines a function `regCoalesce` that does aggressive coalescing directly on `LiveCmmTops`, without using the graph. This isn't used at the moment but has been left in incase we want to rejig the allocator when the new CPS converter comes online.
- [compiler/nativeGen/RegArchBase.hs](/trac/ghc/browser/ghc/compiler/nativeGen/RegArchBase.hs)
- [compiler/nativeGen/RegArchBase.hs](https://gitlab.haskell.org/ghc/ghc/blob/master/compiler/nativeGen/RegArchBase.hs)
Defines utils for calculating whether a register in the conflict graph is trivially colorable, in a generic way which handles aliasing between register classes. This module is not used directly by GHC.
- [compiler/nativeGen/RegArchX86.hs](/trac/ghc/browser/ghc/compiler/nativeGen/RegArchX86.hs)
- [compiler/nativeGen/RegArchX86.hs](https://gitlab.haskell.org/ghc/ghc/blob/master/compiler/nativeGen/RegArchX86.hs)
Contains a description of the aliasing constraints between the register sets on x86. This module is not used directly by GHC.
This diff is collapsed.
......@@ -23,7 +23,7 @@ This file contains functions that actually parse the command line parameters.
Static flags are managed by functions in [compiler/main/StaticFlags.hs](https://gitlab.haskell.org/ghc/ghc/tree/master/ghc/compiler/main/StaticFlags.hs).
Function `parseStaticFlags ::` is an entry point for parsing static flags. It is called by the `main :: IO ()` function of GHC in [ghc/Main.hs](https://gitlab.haskell.org/ghc/ghc/tree/master/ghc/ghc/Main.hs). Two global IORefs are used to parse static flags: `v_opt_C_ready` and `v_opt_C`. These are defined using `GLOBAL_VAR` macro from [compiler/HsVersions.h](/trac/ghc/browser/ghc/compiler/HsVersions.h). First IORef is a flag that checks whether the static flags are parsed at the right time. Initialized to `False`, it is set to `True` after the parsing is done. `v_opt_C` is a `[String]` used to store parsed flags (see `addOpt` and `removeOpt` functions).
Function `parseStaticFlags ::` is an entry point for parsing static flags. It is called by the `main :: IO ()` function of GHC in [ghc/Main.hs](https://gitlab.haskell.org/ghc/ghc/tree/master/ghc/ghc/Main.hs). Two global IORefs are used to parse static flags: `v_opt_C_ready` and `v_opt_C`. These are defined using `GLOBAL_VAR` macro from [compiler/HsVersions.h](https://gitlab.haskell.org/ghc/ghc/blob/master/compiler/HsVersions.h). First IORef is a flag that checks whether the static flags are parsed at the right time. Initialized to `False`, it is set to `True` after the parsing is done. `v_opt_C` is a `[String]` used to store parsed flags (see `addOpt` and `removeOpt` functions).
In [compiler/main/StaticFlags.hs](https://gitlab.haskell.org/ghc/ghc/tree/master/ghc/compiler/main/StaticFlags.hs), `flagsStatic :: [Flag IO]` defines a list of static flags and what actions should be taken when these flags are encountered (see `Flag` data type above). It also contains some helper functions to check whether particular flags have been set. Functions `staticFlags :: [String]` and `packed_staticFlags :: [FastString]` return a list of parsed command line static flags, provided that parsing has been done (checking the value of `v_opt_C_ready`).
......
......@@ -21,7 +21,7 @@ The `CoreSyn` type, and the functions that operate over it, gets an entire direc
- [compiler/coreSyn/CoreLint.hs](https://gitlab.haskell.org/ghc/ghc/tree/master/ghc/compiler/coreSyn/CoreLint.hs): type-check the Core program. This is an incredibly-valuable consistency check, enabled by the flag `-dcore-lint`.
- [compiler/coreSyn/CoreTidy.hs](https://gitlab.haskell.org/ghc/ghc/tree/master/ghc/compiler/coreSyn/CoreTidy.hs): part of the [the CoreTidy pass](commentary/compiler/hsc-main) (the rest is in [compiler/main/TidyPgm.hs](/trac/ghc/browser/ghc/compiler/main/TidyPgm.hs)).
- [compiler/coreSyn/CoreTidy.hs](https://gitlab.haskell.org/ghc/ghc/tree/master/ghc/compiler/coreSyn/CoreTidy.hs): part of the [the CoreTidy pass](commentary/compiler/hsc-main) (the rest is in [compiler/main/TidyPgm.hs](https://gitlab.haskell.org/ghc/ghc/blob/master/compiler/main/TidyPgm.hs)).
- [compiler/coreSyn/CorePrep.hs](https://gitlab.haskell.org/ghc/ghc/tree/master/ghc/compiler/coreSyn/CorePrep.hs): [the CorePrep pass](commentary/compiler/hsc-main)
......
......@@ -12,7 +12,7 @@ At the end of desugaring we run the `simpleOptPgm` function that performs some s
The structure of the Core-to-Core pipeline is determined in the `getCoreToDo` function in the [compiler/simplCore/SimplCore.hs](/trac/ghc/browser/ghc/compiler/simplCore/SimplCore.hs) module. Below is an ordered list of performed optimisations. These are enabled by default with `-O1` and `-O2` unless the description says a specific flag is required. The simplifier, which the pipeline description below often refers to, is described in detail in [the next section](commentary/compiler/core2-core-pipeline#simplifier).
The structure of the Core-to-Core pipeline is determined in the `getCoreToDo` function in the [compiler/simplCore/SimplCore.hs](https://gitlab.haskell.org/ghc/ghc/blob/master/compiler/simplCore/SimplCore.hs) module. Below is an ordered list of performed optimisations. These are enabled by default with `-O1` and `-O2` unless the description says a specific flag is required. The simplifier, which the pipeline description below often refers to, is described in detail in [the next section](commentary/compiler/core2-core-pipeline#simplifier).
- **Static Argument Transformation**: tries to remove redundant arguments to recursive calls, turning them into free variables in those calls. Only enabled with `-fstatic-argument-transformation`. If run this pass is preceded with a "gentle" run of the simplifier.
......
......@@ -44,7 +44,7 @@ Look at the picture first. The yellow boxes are compiler passes, while the blue
- At this point, the data flow forks. First, the tidied program is dumped into an interface file. This part happens in two stages:
* It is **converted to `IfaceSyn`** (defined in [compiler/iface/IfaceSyn.hs](https://gitlab.haskell.org/ghc/ghc/tree/master/ghc/compiler/iface/IfaceSyn.hs) and [compiler/iface/IfaceType.hs](/trac/ghc/browser/ghc/compiler/iface/IfaceType.hs)).
* It is **converted to `IfaceSyn`** (defined in [compiler/iface/IfaceSyn.hs](https://gitlab.haskell.org/ghc/ghc/tree/master/ghc/compiler/iface/IfaceSyn.hs) and [compiler/iface/IfaceType.hs](https://gitlab.haskell.org/ghc/ghc/blob/master/compiler/iface/IfaceType.hs)).
- The `IfaceSyn` is **serialised into a binary output file** ([compiler/iface/BinIface.hs](https://gitlab.haskell.org/ghc/ghc/tree/master/ghc/compiler/iface/BinIface.hs)).
The serialisation does (pretty much) nothing except serialise. All the intelligence is in the `Core`-to-`IfaceSyn` conversion; or, rather, in the reverse of that step.
......
......@@ -26,7 +26,7 @@ Here are some of the things stored in an interface file `M.hi`
- The strictness, arity, and unfolding of exported functions. This is crucial for cross-module optimisation; but it is only included when you compile with `-O`.
The contents of an interface file is the result of serialising the **`IfaceSyn`** family of data types. The data types are in [compiler/iface/IfaceSyn.hs](https://gitlab.haskell.org/ghc/ghc/tree/master/ghc/compiler/iface/IfaceSyn.hs) and [compiler/iface/IfaceType.hs](/trac/ghc/browser/ghc/compiler/iface/IfaceType.hs); the binary serialisation code is in [compiler/iface/BinIface.hs](/trac/ghc/browser/ghc/compiler/iface/BinIface.hs). The definition of a module interface is the **`ModIface`** data type in [compiler/main/HscTypes.hs](/trac/ghc/browser/ghc/compiler/main/HscTypes.hs).
The contents of an interface file is the result of serialising the **`IfaceSyn`** family of data types. The data types are in [compiler/iface/IfaceSyn.hs](https://gitlab.haskell.org/ghc/ghc/tree/master/ghc/compiler/iface/IfaceSyn.hs) and [compiler/iface/IfaceType.hs](https://gitlab.haskell.org/ghc/ghc/blob/master/compiler/iface/IfaceType.hs); the binary serialisation code is in [compiler/iface/BinIface.hs](https://gitlab.haskell.org/ghc/ghc/blob/master/compiler/iface/BinIface.hs). The definition of a module interface is the **`ModIface`** data type in [compiler/main/HscTypes.hs](https://gitlab.haskell.org/ghc/ghc/blob/master/compiler/main/HscTypes.hs).
Details of some of the types involved in GHC's representation of Modules and Interface files can be found [here](commentary/compiler/module-types).
......
......@@ -88,7 +88,7 @@ require complex implementations. We live with this complexity because
Convert from `STG` to an control flow graph `CmmGraph` ([compiler/cmm/ZipCfg.hs](/trac/ghc/browser/ghc/compiler/cmm/ZipCfg.hs), [compiler/cmm/ZipCfgCmmRep.hs](/trac/ghc/browser/ghc/compiler/cmm/ZipCfgCmmRep.hs)). This step is Simon PJ's "new code generator" from September 2007. This conversion may introduce new variables, stack slots, and compile-time constants.
Convert from `STG` to an control flow graph `CmmGraph` ([compiler/cmm/ZipCfg.hs](https://gitlab.haskell.org/ghc/ghc/blob/master/compiler/cmm/ZipCfg.hs), [compiler/cmm/ZipCfgCmmRep.hs](https://gitlab.haskell.org/ghc/ghc/blob/master/compiler/cmm/ZipCfgCmmRep.hs)). This step is Simon PJ's "new code generator" from September 2007. This conversion may introduce new variables, stack slots, and compile-time constants.
```wiki
......@@ -113,7 +113,7 @@ CmmGraph Cmm.Middle Cmm.Last -> CmmGraph I386.Middle I386.Last
```
The `I386.Middle` type represents computational machine instructions; the `I386.Last` type represents control-transfer instructions. The choice of representation is up to the author of the back end, but for continuity with the existing native code generators, we expect to begin by using algebraic data types inspired by the existing definitions in [compiler/nativeGen/MachInstrs.hs](/trac/ghc/browser/ghc/compiler/nativeGen/MachInstrs.hs).
The `I386.Middle` type represents computational machine instructions; the `I386.Last` type represents control-transfer instructions. The choice of representation is up to the author of the back end, but for continuity with the existing native code generators, we expect to begin by using algebraic data types inspired by the existing definitions in [compiler/nativeGen/MachInstrs.hs](https://gitlab.haskell.org/ghc/ghc/blob/master/compiler/nativeGen/MachInstrs.hs).
......
......@@ -27,7 +27,7 @@ There is a new Cmm data type:
- Control-flow blocks: `Block`
- Control-flow graphs: `Graph`
- **`ZipDataFlow`** contains a generic framework for solving dataflow problems over `ZipCfg`. It allows you to define a new optimization simply by defining a lattice of dataflow facts (akin to a specialized logic) and then writing the dataflow-transfer functions found in compiler textbooks. Handing these functions to the dataflow engine produces a new optimization that is not only useful on its own, but that can easily be composed with other optimizations to create an integrated "superoptimization" that is strictly more powerful than any sequence of individual optimizations, no matter how many times they are re-run. The dataflow engine is based on [(Lerner, Grove, and Chambers 2002)](http://citeseer.ist.psu.edu/old/lerner01composing.html); you can find a functional implementation of the dataflow engine presented in [ (Ramsey and Dias 2005)](http://www.cs.tufts.edu/~nr/pubs/zipcfg-abstract.html).
- **[compiler/cmm/ZipCfgCmmRep.hs](/trac/ghc/browser/ghc/compiler/cmm/ZipCfgCmmRep.hs)** instantiates `ZipCfg` for Cmm, by defining types `Middle` and `Last` and using these to instantiate the polymorphic fields of `ZipCfg`. It also defines a bunch of smart constructor (`mkJump`, `mkAssign`, `mkCmmIfThenElse` etc) which make it easy to build `CmmGraph`.
- **[compiler/cmm/ZipCfgCmmRep.hs](https://gitlab.haskell.org/ghc/ghc/blob/master/compiler/cmm/ZipCfgCmmRep.hs)** instantiates `ZipCfg` for Cmm, by defining types `Middle` and `Last` and using these to instantiate the polymorphic fields of `ZipCfg`. It also defines a bunch of smart constructor (`mkJump`, `mkAssign`, `mkCmmIfThenElse` etc) which make it easy to build `CmmGraph`.
- **`CmmExpr`** contains the data types for Cmm expressions, registers, and the like. Here is a fuller description of these types is at [Commentary/Compiler/BackEndTypes](commentary/compiler/back-end-types). It does not depend on the dataflow framework at all.
## Module structure of the new code generator
......
......@@ -25,7 +25,7 @@ GHC draws its information about what packages are installed from one or more pac
## Package-related types
Source files: [compiler/main/PackageConfig.hs](https://gitlab.haskell.org/ghc/ghc/tree/master/ghc/compiler/main/PackageConfig.hs), [compiler/main/Packages.lhs](/trac/ghc/browser/ghc/compiler/main/Packages.lhs)
Source files: [compiler/main/PackageConfig.hs](https://gitlab.haskell.org/ghc/ghc/tree/master/ghc/compiler/main/PackageConfig.hs), [compiler/main/Packages.lhs](https://gitlab.haskell.org/ghc/ghc/blob/master/compiler/main/Packages.lhs)
* **`PackageId`**
......
......@@ -34,7 +34,7 @@ For example, code generated by `deriving` might use an `Orig` to refer to `Prelu
## The `Module` and `ModuleName` types
In GHC, a *module* is uniquely defined by a pair of the module name and the package where the module is defined. The details are in [compiler/basicTypes/Module.hs](https://gitlab.haskell.org/ghc/ghc/tree/master/ghc/compiler/basicTypes/Module.hs) and [compiler/main/PackageConfig.hs](/trac/ghc/browser/ghc/compiler/main/PackageConfig.hs), but here are the key definitions:
In GHC, a *module* is uniquely defined by a pair of the module name and the package where the module is defined. The details are in [compiler/basicTypes/Module.hs](https://gitlab.haskell.org/ghc/ghc/tree/master/ghc/compiler/basicTypes/Module.hs) and [compiler/main/PackageConfig.hs](https://gitlab.haskell.org/ghc/ghc/blob/master/compiler/main/PackageConfig.hs), but here are the key definitions:
```wiki
newtype PackageId = PId FastString
......
......@@ -10,7 +10,7 @@ The rest of this commentary describes code that is not checked in to the HEAD ye
Update: as of 2014-02-12, newer documentation (apparently on the same topic and apparently more up-to-date) is available at [Commentary/Compiler/Demand](commentary/compiler/demand) (I am not an expert on the GHC internals though).
Also, [compiler/basicTypes/NewDemand.lhs](https://gitlab.haskell.org/ghc/ghc/tree/master/ghc/compiler/basicTypes/NewDemand.lhs) is not any more in the sources, replaced by (or renamed to?) [compiler/basicTypes/Demand.lhs](/trac/ghc/browser/ghc/compiler/basicTypes/Demand.lhs).
Also, [compiler/basicTypes/NewDemand.lhs](https://gitlab.haskell.org/ghc/ghc/tree/master/ghc/compiler/basicTypes/NewDemand.lhs) is not any more in the sources, replaced by (or renamed to?) [compiler/basicTypes/Demand.lhs](https://gitlab.haskell.org/ghc/ghc/blob/master/compiler/basicTypes/Demand.lhs).
# The demand analyzer
......
......@@ -56,7 +56,7 @@ staticTickyHdr =
in [compiler/codeGen/CgTicky.hs](https://gitlab.haskell.org/ghc/ghc/tree/master/ghc/compiler/codeGen/CgTicky.hs).
Other relevant functions: `emitTickyCounter` in [compiler/codeGen/CgTicky.hs](https://gitlab.haskell.org/ghc/ghc/tree/master/ghc/compiler/codeGen/CgTicky.hs) (called by `closureCodeBody` in [compiler/codeGen/CgClosure.lhs](/trac/ghc/browser/ghc/compiler/codeGen/CgClosure.lhs)).
Other relevant functions: `emitTickyCounter` in [compiler/codeGen/CgTicky.hs](https://gitlab.haskell.org/ghc/ghc/tree/master/ghc/compiler/codeGen/CgTicky.hs) (called by `closureCodeBody` in [compiler/codeGen/CgClosure.lhs](https://gitlab.haskell.org/ghc/ghc/blob/master/compiler/codeGen/CgClosure.lhs)).
Argh! I spent days tracking down this bug: `idInfoLabelType` in [compiler/cmm/CLabel.hs](https://gitlab.haskell.org/ghc/ghc/tree/master/ghc/compiler/cmm/CLabel.hs) needs to return `DataLabel` for labels of type `RednCount` (i.e., labels for ticky counters.) By default, it was returning `CodeLabel`, which caused the ticky counter labels to get declared with the wrong type in the generated C, which caused C compiler errors.
......@@ -76,7 +76,7 @@ StgWord Main_zdwrepeated_ct[] = {
```
Here, `Main_zdwrepeated_ct` is actually an `StgEntCounter` (this type is declared in [includes/StgTicky.h](https://gitlab.haskell.org/ghc/ghc/tree/master/ghc/includes/StgTicky.h)). The counters get used by `printRegisteredCounterInfo` in [rts/Ticky.c](/trac/ghc/browser/ghc/rts/Ticky.c), which prints out the ticky reports. The counter fields are accessed using offsets defined in [includes/GHCConstants.h](/trac/ghc/browser/ghc/includes/GHCConstants.h) (`oFFSET_StgEntCounter_*`), which in turn get generated from [includes/mkDerivedConstants.c](/trac/ghc/browser/ghc/includes/mkDerivedConstants.c) (change it and then run `make` in `includes/`.
Here, `Main_zdwrepeated_ct` is actually an `StgEntCounter` (this type is declared in [includes/StgTicky.h](https://gitlab.haskell.org/ghc/ghc/tree/master/ghc/includes/StgTicky.h)). The counters get used by `printRegisteredCounterInfo` in [rts/Ticky.c](https://gitlab.haskell.org/ghc/ghc/blob/master/rts/Ticky.c), which prints out the ticky reports. The counter fields are accessed using offsets defined in [includes/GHCConstants.h](https://gitlab.haskell.org/ghc/ghc/blob/master/includes/GHCConstants.h) (`oFFSET_StgEntCounter_*`), which in turn get generated from [includes/mkDerivedConstants.c](https://gitlab.haskell.org/ghc/ghc/blob/master/includes/mkDerivedConstants.c) (change it and then run `make` in `includes/`.
\<s\>Note that the first 3 fields of the counters are 16-bit ints and so the generated ticky-counter registration code has to reflect that (I fixed a bug where the first field was getting treated as a 32-bit int.)\</s\> I modified the `StgEntCounter` type so that all fields are `StgWord`s, because it seems that the code generator can't cope with anything else anyway (i.e., in the declaration above, `Main_zdwrepeated_ct[]` is an array of `StgWord`s, even though the C type declaration implies that some fields are halfwords.)
......
......@@ -4,7 +4,7 @@
Probably the most important phase in the frontend is the type checker, which is located at [compiler/typecheck/](https://gitlab.haskell.org/ghc/ghc/tree/master/ghc/compiler/typecheck/). GHC type checks programs in their original Haskell form before the desugarer converts them into Core code. This complicates the type checker as it has to handle the much more verbose Haskell AST, but it improves error messages, as those message are based on the same structure that the user sees.
GHC defines the abstract syntax of Haskell programs in [compiler/hsSyn/HsSyn.hs](https://gitlab.haskell.org/ghc/ghc/tree/master/ghc/compiler/hsSyn/HsSyn.hs) using a structure that abstracts over the concrete representation of bound occurences of identifiers and patterns. The module [compiler/typecheck/TcHsSyn.hs](/trac/ghc/browser/ghc/compiler/typecheck/TcHsSyn.hs) defines a number of helper function required by the type checker. Note that the type [compiler/typecheck/TcRnTypes.hs](/trac/ghc/browser/ghc/compiler/typecheck/TcRnTypes.hs).`TcId` used to represent identifiers in some signatures during type checking is, in fact, nothing but a synonym for a [plain Id](commentary/compiler/entity-types#type-variables-and-term-variables).
GHC defines the abstract syntax of Haskell programs in [compiler/hsSyn/HsSyn.hs](https://gitlab.haskell.org/ghc/ghc/tree/master/ghc/compiler/hsSyn/HsSyn.hs) using a structure that abstracts over the concrete representation of bound occurences of identifiers and patterns. The module [compiler/typecheck/TcHsSyn.hs](https://gitlab.haskell.org/ghc/ghc/blob/master/compiler/typecheck/TcHsSyn.hs) defines a number of helper function required by the type checker. Note that the type [compiler/typecheck/TcRnTypes.hs](https://gitlab.haskell.org/ghc/ghc/blob/master/compiler/typecheck/TcRnTypes.hs).`TcId` used to represent identifiers in some signatures during type checking is, in fact, nothing but a synonym for a [plain Id](commentary/compiler/entity-types#type-variables-and-term-variables).
It is also noteworthy, that the representations of types changes during type checking from `HsType` to `TypeRep.Type`. The latter is a [hybrid type](commentary/compiler/type-type) representation that is used to type Core, but still contains sufficient information to recover source types. In particular, the type checker maintains and compares types in their `Type` form.
......@@ -84,7 +84,7 @@ Inside this big knot, the first main operation is kind checking, which again inv
### Types Variables and Zonking
During type checking type variables are represented by mutable variables - cf. the [variable story](http://darcs.haskell.org/ghc/docs/comm/the-beast/vars.html#TyVar) (TODO Point at new commentary equivalent). Consequently, unification can instantiate type variables by updating those mutable variables. This process of instantiation is (for reasons that elude me) called [zonking](http://dictionary.reference.com/browse/zonk) in GHC's sources. The zonking routines for the various forms of Haskell constructs are responsible for most of the code in the module [compiler/typecheck/TcHsSyn.hs](https://gitlab.haskell.org/ghc/ghc/tree/master/ghc/compiler/typecheck/TcHsSyn.hs), whereas the routines that actually operate on mutable types are defined in [compiler/typecheck/TcMType.hs](/trac/ghc/browser/ghc/compiler/typecheck/TcMType.hs); this includes the zonking of type variables and type terms, routines to create mutable structures and update them as well as routines that check constraints, such as that type variables in function signatures have not been instantiated during type checking. The actual type unification routine is `uTys` in the module [compiler/typecheck/TcUnify.hs](/trac/ghc/browser/ghc/compiler/typecheck/TcUnify.hs).
During type checking type variables are represented by mutable variables - cf. the [variable story](http://darcs.haskell.org/ghc/docs/comm/the-beast/vars.html#TyVar) (TODO Point at new commentary equivalent). Consequently, unification can instantiate type variables by updating those mutable variables. This process of instantiation is (for reasons that elude me) called [zonking](http://dictionary.reference.com/browse/zonk) in GHC's sources. The zonking routines for the various forms of Haskell constructs are responsible for most of the code in the module [compiler/typecheck/TcHsSyn.hs](https://gitlab.haskell.org/ghc/ghc/tree/master/ghc/compiler/typecheck/TcHsSyn.hs), whereas the routines that actually operate on mutable types are defined in [compiler/typecheck/TcMType.hs](https://gitlab.haskell.org/ghc/ghc/blob/master/compiler/typecheck/TcMType.hs); this includes the zonking of type variables and type terms, routines to create mutable structures and update them as well as routines that check constraints, such as that type variables in function signatures have not been instantiated during type checking. The actual type unification routine is `uTys` in the module [compiler/typecheck/TcUnify.hs](https://gitlab.haskell.org/ghc/ghc/blob/master/compiler/typecheck/TcUnify.hs).
All type variables that may be instantiated (those in signatures may not), but haven't been instantiated during type checking, are zonked to `()`, so that after type checking all mutable variables have been eliminated.
......@@ -104,7 +104,7 @@ As explained in [compiler/typecheck/TcType.hs](https://gitlab.haskell.org/ghc/gh
### Type Checking Environment
During type checking, GHC maintains a *type environment* whose type definitions are fixed in the module [compiler/typecheck/TcRnTypes.hs](https://gitlab.haskell.org/ghc/ghc/tree/master/ghc/compiler/typecheck/TcRnTypes.hs) with the operations defined in [compiler/typecheck/TcEnv.hs](/trac/ghc/browser/ghc/compiler/typecheck/TcEnv.hs). Among other things, the environment contains all imported and local instances as well as a list of *global* entities (imported and local types and classes together with imported identifiers) and *local* entities (locally defined identifiers). This environment is threaded through the [type checking monad](commentary/compiler/tc-rn-monad).
During type checking, GHC maintains a *type environment* whose type definitions are fixed in the module [compiler/typecheck/TcRnTypes.hs](https://gitlab.haskell.org/ghc/ghc/tree/master/ghc/compiler/typecheck/TcRnTypes.hs) with the operations defined in [compiler/typecheck/TcEnv.hs](https://gitlab.haskell.org/ghc/ghc/blob/master/compiler/typecheck/TcEnv.hs). Among other things, the environment contains all imported and local instances as well as a list of *global* entities (imported and local types and classes together with imported identifiers) and *local* entities (locally defined identifiers). This environment is threaded through the [type checking monad](commentary/compiler/tc-rn-monad).
### Expressions
......
......@@ -22,7 +22,7 @@ Note, that "one compiler invocation" is not the same as the compilation of a sin
This is also the reasons why `OccName`s are *not* ordered based on the `Unique`s of their underlying `FastString`s, but rather *lexicographically* (see [compiler/basicTypes/OccName.lhs](/trac/ghc/browser/ghc/compiler/basicTypes/OccName.lhs) for details).
This is also the reasons why `OccName`s are *not* ordered based on the `Unique`s of their underlying `FastString`s, but rather *lexicographically* (see [compiler/basicTypes/OccName.lhs](https://gitlab.haskell.org/ghc/ghc/blob/master/compiler/basicTypes/OccName.lhs) for details).
>
......@@ -134,7 +134,7 @@ mkUniqueGrimily i = MkUnique (iUnbox i)
```
this separation of concerns leaked out to [compiler/basicTypes/UniqSupply.lhs](/trac/ghc/browser/ghc/compiler/basicTypes/UniqSupply.lhs), because its `Int` argument is the *entire* `Unique` and not just the integer part 'under' the domain character.
this separation of concerns leaked out to [compiler/basicTypes/UniqSupply.lhs](https://gitlab.haskell.org/ghc/ghc/blob/master/compiler/basicTypes/UniqSupply.lhs), because its `Int` argument is the *entire* `Unique` and not just the integer part 'under' the domain character.
>
......
......@@ -16,7 +16,7 @@ A **Wired-in thing** is fully known to GHC. Most of these are `TyCon`s such as
All [primitive types](commentary/compiler/type-type#classifying-types) are wired-in things, and have wired-in `Name`s. The primitive types (and their `Names`) are all defined in [compiler/prelude/TysPrim.hs](https://gitlab.haskell.org/ghc/ghc/tree/master/ghc/compiler/prelude/TysPrim.hs).
The non-primitive wired-in type constructors are defined in [compiler/prelude/TysWiredIn.hs](https://gitlab.haskell.org/ghc/ghc/tree/master/ghc/compiler/prelude/TysWiredIn.hs). There are a handful of wired-in `Id`s in [compiler/basicTypes/MkId.hs](/trac/ghc/browser/ghc/compiler/basicTypes/MkId.hs). There are no wired-in classes (they are too complicated).
The non-primitive wired-in type constructors are defined in [compiler/prelude/TysWiredIn.hs](https://gitlab.haskell.org/ghc/ghc/tree/master/ghc/compiler/prelude/TysWiredIn.hs). There are a handful of wired-in `Id`s in [compiler/basicTypes/MkId.hs](https://gitlab.haskell.org/ghc/ghc/blob/master/compiler/basicTypes/MkId.hs). There are no wired-in classes (they are too complicated).
All the non-primitive wired-in things are *also* defined in GHC's libraries, because even though GHC knows about them we still need to generate code for them. For example, `Bool` is a wired-in type constructor, but it is still defined in `GHC.Base` because we need the info table etc for the data constructors. Arbitrarily bad things will happen if the wired-in definition in [compiler/prelude/TysWiredIn.hs](https://gitlab.haskell.org/ghc/ghc/tree/master/ghc/compiler/prelude/TysWiredIn.hs) differs from that in the library module.
......
......@@ -65,7 +65,7 @@ plusInteger :: Integer -> Integer -> Integer
While `Integer`s aren't "machine literals" like the other `Core` `Literal` constructors, it is more convenient when writing constant folding RULES to pretend that they are literals rather than having to understand their concrete representation. (Especially as the concrete representation varies from package to package.) We also carry around a `Type`, representing the `Integer` type, in the constructor, as we need access to it in a few functions (e.g. `literalType`).
- **Constant folding**. There are many constant-folding optimisations for `Integer` expressed as built-in rules in [compiler/prelude/PrelRules.lhs](https://gitlab.haskell.org/ghc/ghc/tree/master/ghc/compiler/prelude/PrelRules.lhs); look at `builtinIntegerRules`. All of the types and functions in the `Integer` interface have built-in names, e.g. `plusIntegerName`, defined in [compiler/prelude/PrelNames.lhs](/trac/ghc/browser/ghc/compiler/prelude/PrelNames.lhs) and included in `basicKnownKeyNames`. This allows us to match on all of the functions in `builtinIntegerRules` in [compiler/prelude/PrelRules.lhs](/trac/ghc/browser/ghc/compiler/prelude/PrelRules.lhs), so we can constant-fold Integer expressions. An important thing about constant folding of Integer divisions is that they depend on inlining. Here's a fragment of `Integral Integer` instance definition from `libraries/base/GHC/Real.lhs`:
- **Constant folding**. There are many constant-folding optimisations for `Integer` expressed as built-in rules in [compiler/prelude/PrelRules.lhs](https://gitlab.haskell.org/ghc/ghc/tree/master/ghc/compiler/prelude/PrelRules.lhs); look at `builtinIntegerRules`. All of the types and functions in the `Integer` interface have built-in names, e.g. `plusIntegerName`, defined in [compiler/prelude/PrelNames.lhs](https://gitlab.haskell.org/ghc/ghc/blob/master/compiler/prelude/PrelNames.lhs) and included in `basicKnownKeyNames`. This allows us to match on all of the functions in `builtinIntegerRules` in [compiler/prelude/PrelRules.lhs](https://gitlab.haskell.org/ghc/ghc/blob/master/compiler/prelude/PrelRules.lhs), so we can constant-fold Integer expressions. An important thing about constant folding of Integer divisions is that they depend on inlining. Here's a fragment of `Integral Integer` instance definition from `libraries/base/GHC/Real.lhs`:
```haskell
instance Integral Integer where
......
......@@ -66,7 +66,7 @@ There are some important functions if you are tracing how things get from GHC to
# The driver pipeline
The driver pipeline consist of a couple of phases that call other programs and generate a series of intermediate files. Code responsible for managing the order of phases is in [compiler/main/DriverPhases.hs](https://gitlab.haskell.org/ghc/ghc/tree/master/ghc/compiler/main/DriverPhases.hs), while managing the driver pipeline as a whole is coded in [compiler/main/DriverPipeline.hs](/trac/ghc/browser/ghc/compiler/main/DriverPipeline.hs). Note that driver pipeline is not the same thing as compilation pipeline: the latter is part of the former.
The driver pipeline consist of a couple of phases that call other programs and generate a series of intermediate files. Code responsible for managing the order of phases is in [compiler/main/DriverPhases.hs](https://gitlab.haskell.org/ghc/ghc/tree/master/ghc/compiler/main/DriverPhases.hs), while managing the driver pipeline as a whole is coded in [compiler/main/DriverPipeline.hs](https://gitlab.haskell.org/ghc/ghc/blob/master/compiler/main/DriverPipeline.hs). Note that driver pipeline is not the same thing as compilation pipeline: the latter is part of the former.
Let's take a look at the overall structure of the driver pipeline. When we compile `Foo.hs` or `Foo.lhs` ("lhs" extension means that Literate Haskell is being used) the following phases are being called (some of them depending on additional conditions like file extensions or enabled flags):
......
......@@ -35,4 +35,4 @@ In the compiler's source code, you may make use of the following CPP symbols:
where *xxx* is the appropriate value: eg. `i386_TARGET_ARCH`. However **GHC is moving away from using CPP for this purpose** in many cases due to the problems it creates with supporting cross compilation.
So instead of it the new plan is to always build GHC as a cross compiler and select the appropriate values and backend code generator to run and runtime. For this purpose there is the Platform module ([compiler/utils/Platform.hs](https://gitlab.haskell.org/ghc/ghc/tree/master/ghc/compiler/utils/Platform.hs)). That contains various methods for querying the DynFlags ([compiler/main/DynFlags.hs](/trac/ghc/browser/ghc/compiler/main/DynFlags.hs)) value for what platform GHC is currently compiling for. You should use these when appropriate over the CPP methods.
So instead of it the new plan is to always build GHC as a cross compiler and select the appropriate values and backend code generator to run and runtime. For this purpose there is the Platform module ([compiler/utils/Platform.hs](https://gitlab.haskell.org/ghc/ghc/tree/master/ghc/compiler/utils/Platform.hs)). That contains various methods for querying the DynFlags ([compiler/main/DynFlags.hs](https://gitlab.haskell.org/ghc/ghc/blob/master/compiler/main/DynFlags.hs)) value for what platform GHC is currently compiling for. You should use these when appropriate over the CPP methods.
......@@ -139,15 +139,15 @@ run `make clean` so the build system will pick up changes to the generated `GHC.
In addition, if new primtypes are being added, the following files need to be updated:
- [utils/genprimopcode/Main.hs](/trac/ghc/browser/ghc/utils/genprimopcode/Main.hs) -- extend ppType :: Type -\> String function
- [utils/genprimopcode/Main.hs](https://gitlab.haskell.org/ghc/ghc/blob/master/utils/genprimopcode/Main.hs) -- extend ppType :: Type -\> String function
- [compiler/prelude/PrelNames.hs](/trac/ghc/browser/ghc/compiler/prelude/PrelNames.hs) -- add a new unique id using mkPreludeTyConUnique
- [compiler/prelude/PrelNames.hs](https://gitlab.haskell.org/ghc/ghc/blob/master/compiler/prelude/PrelNames.hs) -- add a new unique id using mkPreludeTyConUnique
- [compiler/prelude/TysPrim.hs](https://gitlab.haskell.org/ghc/ghc/tree/master/ghc/compiler/prelude/TysPrim.hs) -- there are a raft of changes here; you need to create `*PrimTy`, `*PrimTyCon` and `*PrimTyConName` variables. The most important thing to make sure you get right is when you make a PrimTyCon, you pick the correct `PrimRep` for your type. For example, if you’ve introduced a new GC'able object, you should use `PtrRep`; however, if it's a pointer that shouldn't be GC'd, you should use `AddrRep` instead. The full list is in [compiler/types/TyCon.hs](/trac/ghc/browser/ghc/compiler/types/TyCon.hs)
- [compiler/prelude/TysPrim.hs](https://gitlab.haskell.org/ghc/ghc/tree/master/ghc/compiler/prelude/TysPrim.hs) -- there are a raft of changes here; you need to create `*PrimTy`, `*PrimTyCon` and `*PrimTyConName` variables. The most important thing to make sure you get right is when you make a PrimTyCon, you pick the correct `PrimRep` for your type. For example, if you’ve introduced a new GC'able object, you should use `PtrRep`; however, if it's a pointer that shouldn't be GC'd, you should use `AddrRep` instead. The full list is in [compiler/types/TyCon.hs](https://gitlab.haskell.org/ghc/ghc/blob/master/compiler/types/TyCon.hs)
See also [AddingNewPrimitiveOperations](adding-new-primitive-operations), a blow-by-blow description of the process for adding a new out-of-line primop from someone who went through the process.
......
......@@ -16,11 +16,11 @@ Cost-center profiling in GHC, e.g. of SCCs, consists of the following components
- Data-structures for representing cost-centres in [compiler/profiling/CostCentre.hs](https://gitlab.haskell.org/ghc/ghc/tree/master/ghc/compiler/profiling/CostCentre.hs).
- Front-end support in [compiler/deSugar/DsExpr.hs](https://gitlab.haskell.org/ghc/ghc/tree/master/ghc/compiler/deSugar/DsExpr.hs), for converting `SCC` pragma into the `Tick` constructor in Core.
- Modifications to optimization behavior in [compiler/coreSyn/CoreUtils.hs](https://gitlab.haskell.org/ghc/ghc/tree/master/ghc/compiler/coreSyn/CoreUtils.hs) and [compiler/coreSyn/CorePrep.hs](/trac/ghc/browser/ghc/compiler/coreSyn/CorePrep.hs) to prevent optimizations which would result in misleading profile information. Most of this is to handle the fact that SCCs also count entries (tickishCounts, also applies to [Commentary/Hpc](commentary/hpc)); otherwise the only relevant optimization is avoiding floating expressions out of SCCs. Note that the simplifier also has "ticks" (so it can decide when to stop optimizing); these are not the same thing at all.
- Modifications to optimization behavior in [compiler/coreSyn/CoreUtils.hs](https://gitlab.haskell.org/ghc/ghc/tree/master/ghc/compiler/coreSyn/CoreUtils.hs) and [compiler/coreSyn/CorePrep.hs](https://gitlab.haskell.org/ghc/ghc/blob/master/compiler/coreSyn/CorePrep.hs) to prevent optimizations which would result in misleading profile information. Most of this is to handle the fact that SCCs also count entries (tickishCounts, also applies to [Commentary/Hpc](commentary/hpc)); otherwise the only relevant optimization is avoiding floating expressions out of SCCs. Note that the simplifier also has "ticks" (so it can decide when to stop optimizing); these are not the same thing at all.
- The `StgSCC` constructor in STG, and code generation for it [compiler/codeGen/StgCmmProf.hs](https://gitlab.haskell.org/ghc/ghc/tree/master/ghc/compiler/codeGen/StgCmmProf.hs)
- A pass over STG in [compiler/profiling/SCCfinal.hs](https://gitlab.haskell.org/ghc/ghc/tree/master/ghc/compiler/profiling/SCCfinal.hs) to collect cost centres so that they can be statically declared by [compiler/profiling/ProfInit.hs](/trac/ghc/browser/ghc/compiler/profiling/ProfInit.hs), and add extra SCCs in the case of `-fprof-auto`; see also [compiler/profiling/NOTES](/trac/ghc/browser/ghc/compiler/profiling/NOTES)
- A pass over STG in [compiler/profiling/SCCfinal.hs](https://gitlab.haskell.org/ghc/ghc/tree/master/ghc/compiler/profiling/SCCfinal.hs) to collect cost centres so that they can be statically declared by [compiler/profiling/ProfInit.hs](https://gitlab.haskell.org/ghc/ghc/blob/master/compiler/profiling/ProfInit.hs), and add extra SCCs in the case of `-fprof-auto`; see also [compiler/profiling/NOTES](https://gitlab.haskell.org/ghc/ghc/blob/master/compiler/profiling/NOTES)
- Code-generation for setting labels found in [compiler/codeGen/StgCmmProf.hs](https://gitlab.haskell.org/ghc/ghc/tree/master/ghc/compiler/codeGen/StgCmmProf.hs), in particular saving and restoring CC labels and well as counting ticks; note that cost-centres even get their own constructor in C-- as CC_Labels (cost-centre labels).
- Runtime support for initializing and manipulating the actual runtime `CostCentre` structs which store information, in [rts/Profiling.c](https://gitlab.haskell.org/ghc/ghc/tree/master/ghc/rts/Profiling.c); headers are located in [includes/rts/prof/CCS.h](/trac/ghc/browser/ghc/includes/rts/prof/CCS.h)
- Runtime support for initializing and manipulating the actual runtime `CostCentre` structs which store information, in [rts/Profiling.c](https://gitlab.haskell.org/ghc/ghc/tree/master/ghc/rts/Profiling.c); headers are located in [includes/rts/prof/CCS.h](https://gitlab.haskell.org/ghc/ghc/blob/master/includes/rts/prof/CCS.h)
## Ticky-ticky profiling
......
......@@ -4,13 +4,13 @@
# Kirsten's sketchy notes on getting ticky to work
Macros for bumping ticky counters are now defined in [includes/Cmm.h](https://gitlab.haskell.org/ghc/ghc/tree/master/ghc/includes/Cmm.h). Currently, code compiled with the `-fticky-ticky` flag fails to link because the macros rely on counter variables (things with names like `ENT_DYN_IND_ctr` being declared, but there are actually no declarations for them. I'll add those declarations to [includes/RtsExternal.h](/trac/ghc/browser/ghc/includes/RtsExternal.h) so I can get something working. Really, there should be something that automatically generates both the macros that are in [includes/Cmm.h](/trac/ghc/browser/ghc/includes/Cmm.h) and the declarations for the corresponding variables, so that they stay in sync.
Macros for bumping ticky counters are now defined in [includes/Cmm.h](https://gitlab.haskell.org/ghc/ghc/tree/master/ghc/includes/Cmm.h). Currently, code compiled with the `-fticky-ticky` flag fails to link because the macros rely on counter variables (things with names like `ENT_DYN_IND_ctr` being declared, but there are actually no declarations for them. I'll add those declarations to [includes/RtsExternal.h](https://gitlab.haskell.org/ghc/ghc/blob/master/includes/RtsExternal.h) so I can get something working. Really, there should be something that automatically generates both the macros that are in [includes/Cmm.h](https://gitlab.haskell.org/ghc/ghc/blob/master/includes/Cmm.h) and the declarations for the corresponding variables, so that they stay in sync.
Actually, maybe it would make more sense to add a new file, `RtsTicky.h` or something, which contains only ticky counter declarations (the same declarations that still exist in [includes/StgTicky.h](https://gitlab.haskell.org/ghc/ghc/tree/master/ghc/includes/StgTicky.h), which isn't used anymore), and that include that from [includes/RtsExternal.h](/trac/ghc/browser/ghc/includes/RtsExternal.h).
Actually, maybe it would make more sense to add a new file, `RtsTicky.h` or something, which contains only ticky counter declarations (the same declarations that still exist in [includes/StgTicky.h](https://gitlab.haskell.org/ghc/ghc/tree/master/ghc/includes/StgTicky.h), which isn't used anymore), and that include that from [includes/RtsExternal.h](https://gitlab.haskell.org/ghc/ghc/blob/master/includes/RtsExternal.h).
No -- put actual declarations for counter variables in another file, `TickyCounters.h` or something, and include that only from [rts/Ticky.c](https://gitlab.haskell.org/ghc/ghc/tree/master/ghc/rts/Ticky.c); put *extern* declarations for those counters in `RtsTicky.h`, still included from [includes/RtsExternal.h](/trac/ghc/browser/ghc/includes/RtsExternal.h). Then later we can automatically generate both `RtsTicky.h` and `TickyCounters.h`. The reason for this is that the ticky **macros** are all over the place and they refer to the ticky counters, so the ticky counters have to be **declared** someplace that everyone includes, but of course the actual initializations only need to happen in one place. (Maybe there's a better way to do this...)
No -- put actual declarations for counter variables in another file, `TickyCounters.h` or something, and include that only from [rts/Ticky.c](https://gitlab.haskell.org/ghc/ghc/tree/master/ghc/rts/Ticky.c); put *extern* declarations for those counters in `RtsTicky.h`, still included from [includes/RtsExternal.h](https://gitlab.haskell.org/ghc/ghc/blob/master/includes/RtsExternal.h). Then later we can automatically generate both `RtsTicky.h` and `TickyCounters.h`. The reason for this is that the ticky **macros** are all over the place and they refer to the ticky counters, so the ticky counters have to be **declared** someplace that everyone includes, but of course the actual initializations only need to happen in one place. (Maybe there's a better way to do this...)
No, there don't need to be two files; I was confused. Just `TickyCounters.h`.
......
# GHC Commentary: What the hell is a `.cmm` file?
A `.cmm` file is rather like C--. The syntax is almost C-- (a few constructs are missing), and it is augmented with some macros that are expanded by GHC's code generator (eg. `INFO_TABLE()`). A `.cmm` file is compiled by GHC itself: the syntax is parsed by [compiler/cmm/CmmParse.y](https://gitlab.haskell.org/ghc/ghc/tree/master/ghc/compiler/cmm/CmmParse.y) and [compiler/cmm/CmmLex.x](/trac/ghc/browser/ghc/compiler/cmm/CmmLex.x) into the [Cmm](commentary/compiler/cmm-type) data type, where it is then passed through one of the [back-ends](commentary/compiler/backends).
A `.cmm` file is rather like C--. The syntax is almost C-- (a few constructs are missing), and it is augmented with some macros that are expanded by GHC's code generator (eg. `INFO_TABLE()`). A `.cmm` file is compiled by GHC itself: the syntax is parsed by [compiler/cmm/CmmParse.y](https://gitlab.haskell.org/ghc/ghc/tree/master/ghc/compiler/cmm/CmmParse.y) and [compiler/cmm/CmmLex.x](https://gitlab.haskell.org/ghc/ghc/blob/master/compiler/cmm/CmmLex.x) into the [Cmm](commentary/compiler/cmm-type) data type, where it is then passed through one of the [back-ends](commentary/compiler/backends).
We use the C preprocessor on `.cmm` files, making extensive use of macros to make writing this low-level code a bit less tedious and error-prone. Most of our C-- macros are in [includes/Cmm.h](https://gitlab.haskell.org/ghc/ghc/tree/master/ghc/includes/Cmm.h). One useful fact about the macros is `P_` is an alias for `gcptr`, and you should not use it for non-garbage-collected pointers.
......
......@@ -4,7 +4,7 @@
Files [rts/Adjustor.c](/trac/ghc/browser/ghc/rts/Adjustor.c) [rts/AdjustorAsm.S](/trac/ghc/browser/ghc/rts/AdjustorAsm.S).
Files [rts/Adjustor.c](https://gitlab.haskell.org/ghc/ghc/blob/master/rts/Adjustor.c) [rts/AdjustorAsm.S](https://gitlab.haskell.org/ghc/ghc/blob/master/rts/AdjustorAsm.S).
......
# Function Calls
Source files: [rts/Apply.h](https://gitlab.haskell.org/ghc/ghc/tree/master/ghc/rts/Apply.h), [rts/Apply.cmm](/trac/ghc/browser/ghc/rts/Apply.cmm)
Source files: [rts/Apply.h](https://gitlab.haskell.org/ghc/ghc/tree/master/ghc/rts/Apply.h), [rts/Apply.cmm](https://gitlab.haskell.org/ghc/ghc/blob/master/rts/Apply.cmm)
Dealing with calls is by far the most complicated bit of the execution model, and hence of the code generator. GHC uses an *eval/apply* strategy for compiling function calls; all the details of the design are in the paper [Making a fast curry: push/enter vs. eval/apply for higher-order languages](http://simonmar.github.io/bib/papers/evalapplyjfp06.pdf).
......
# Haskell Excecution: Registers
Source files: [includes/stg/Regs.h](https://gitlab.haskell.org/ghc/ghc/tree/master/ghc/includes/stg/Regs.h), [includes/stg/MachRegs.h](/trac/ghc/browser/ghc/includes/stg/MachRegs.h)
Source files: [includes/stg/Regs.h](https://gitlab.haskell.org/ghc/ghc/tree/master/ghc/includes/stg/Regs.h), [includes/stg/MachRegs.h](https://gitlab.haskell.org/ghc/ghc/blob/master/includes/stg/MachRegs.h)
During execution of Haskell code the following (virtual) registers are always valid:
......
......@@ -2,7 +2,7 @@
Source files: [rts/Updates.h](/trac/ghc/browser/ghc/rts/Updates.h), [rts/Updates.cmm](/trac/ghc/browser/ghc/rts/Updates.cmm)
Source files: [rts/Updates.h](https://gitlab.haskell.org/ghc/ghc/blob/master/rts/Updates.h), [rts/Updates.cmm](https://gitlab.haskell.org/ghc/ghc/blob/master/rts/Updates.cmm)
---
......
# Sanity Checking
Source code: [rts/sm/Sanity.c](https://gitlab.haskell.org/ghc/ghc/tree/master/ghc/rts/sm/Sanity.c), [rts/sm/Sanity.h](/trac/ghc/browser/ghc/rts/sm/Sanity.h).
Source code: [rts/sm/Sanity.c](https://gitlab.haskell.org/ghc/ghc/tree/master/ghc/rts/sm/Sanity.c), [rts/sm/Sanity.h](https://gitlab.haskell.org/ghc/ghc/blob/master/rts/sm/Sanity.h).
The purpose of sanity checking is to catch bugs in the RTS as early as possible; if the program is going to crash, we want it to crash as soon as possible after the error occurred. The problem with debugging the RTS is that heap corruption can go unnoticed through several GC cycles, making it particularly difficult to trace back to the erroneous code.
......
......@@ -22,7 +22,7 @@ We begin by discussing the basic abstractions used in the scheduler.
Source files: [includes/rts/OSThreads.h](https://gitlab.haskell.org/ghc/ghc/tree/master/ghc/includes/rts/OSThreads.h),
[rts/win32/OSThreads.c](https://gitlab.haskell.org/ghc/ghc/tree/master/ghc/rts/win32/OSThreads.c), [rts/posix/OSThreads.c](/trac/ghc/browser/ghc/rts/posix/OSThreads.c)
[rts/win32/OSThreads.c](https://gitlab.haskell.org/ghc/ghc/tree/master/ghc/rts/win32/OSThreads.c), [rts/posix/OSThreads.c](https://gitlab.haskell.org/ghc/ghc/blob/master/rts/posix/OSThreads.c)
We assume that the OS provides some kind of native threads, and for
......@@ -68,7 +68,7 @@ two kinds of Haskell thread:
thread are made by an arbitrary OS thread.
Initialization of TSOs is handled in `createThread` in [rts/Threads.c](https://gitlab.haskell.org/ghc/ghc/tree/master/ghc/rts/Threads.c); this function is in turn invoked by `createGenThread`, `createIOThread` and `createStrictIOThread` in [rts/RtsAPI.c](/trac/ghc/browser/ghc/rts/RtsAPI.c). These functions setup the initial stack state, which controls what the thread executes when it actually gets run. These functions are the ones invoked by the `fork#` and other primops (recall entry-points for primops are located in [rts/PrimOps.cmm](/trac/ghc/browser/ghc/rts/PrimOps.cmm)).
Initialization of TSOs is handled in `createThread` in [rts/Threads.c](https://gitlab.haskell.org/ghc/ghc/tree/master/ghc/rts/Threads.c); this function is in turn invoked by `createGenThread`, `createIOThread` and `createStrictIOThread` in [rts/RtsAPI.c](https://gitlab.haskell.org/ghc/ghc/blob/master/rts/RtsAPI.c). These functions setup the initial stack state, which controls what the thread executes when it actually gets run. These functions are the ones invoked by the `fork#` and other primops (recall entry-points for primops are located in [rts/PrimOps.cmm](https://gitlab.haskell.org/ghc/ghc/blob/master/rts/PrimOps.cmm)).
Being garbage collected has two major implications for TSOs. First, TSOs are not GC roots, so they will get GC'd if there is nothing holding on to them (e.g. [in the case of deadlock](http://blog.ezyang.com/2011/07/blockedindefinitelyonmvar)), and their space is not automatically reclaimed when they finish executing (so `ThreadId` can cause memory leaks}}}. Usually, a TSO will be retained by a Capability’s run queue (a GC root), or in the list of waiting threads of some concurrency variable, e.g. an MVar. Second, a TSO must be considered a mutable object, and is thus subject to the conventional GC write barriers necessary for any mutable object in a generational garbage collector. The `dirty` bit tracks whether or not a TSO has been modified; it is always set when a thread is run and also when any of the pointer fields on a TSO are modified.
......@@ -76,7 +76,7 @@ Being garbage collected has two major implications for TSOs. First, TSOs are not
## Tasks
Source files: [rts/Task.h](https://gitlab.haskell.org/ghc/ghc/tree/master/ghc/rts/Task.h), [rts/Task.c](/trac/ghc/browser/ghc/rts/Task.c)
Source files: [rts/Task.h](https://gitlab.haskell.org/ghc/ghc/tree/master/ghc/rts/Task.h), [rts/Task.c](https://gitlab.haskell.org/ghc/ghc/blob/master/rts/Task.c)
A Task is a further layer of abstraction over an OS thread. One `Task` structure is created for each OS thread known to the runtime. To get the `Task` associated with with the current OS thread, use the function `myTask`:
......@@ -122,7 +122,7 @@ A task has a small cache of spare `InCall` structures so that it can allocate a
## Capabilities
Source files: [rts/Capability.h](https://gitlab.haskell.org/ghc/ghc/tree/master/ghc/rts/Capability.h), [rts/Capability.c](/trac/ghc/browser/ghc/rts/Capability.c)
Source files: [rts/Capability.h](https://gitlab.haskell.org/ghc/ghc/tree/master/ghc/rts/Capability.h), [rts/Capability.c](https://gitlab.haskell.org/ghc/ghc/blob/master/rts/Capability.c)
A Capability is a *virtual CPU* for executing Haskell code. The
......@@ -243,7 +243,7 @@ scheduler(cap)
### The run queue
The run queue is at the heart of the scheduler, as any runnable thread will hit the run queue before the scheduler actually pops it off the queue and runs it. There’s one per capability [rts/Capability.h](/trac/ghc/browser/ghc/rts/Capability.h) (in the bad old days, there was a global run queue, but this performed badly for multithreaded processes), and it is implemented as a doubly-linked list `run_queue_hd` and `run_queue_tl`. (It is doubly linked due to ticket #3838). The head and tail pointers mean that the queue is actually a deque: this is important because the scheduler will often have to handle threads that were interrupted in some way, and should let the threads get back on. The links themselves are on the TSOs and modified with `setTSOLink` and `setTSOPrev`, so modifying the queue dirties the TSOs involved. Otherwise, the run queue is exclusively owned by the scheduler. If there are idle capabilities and if we have more than one thread left in our run queue, threads will be pushed to other queues with `schedulePushWork`.
The run queue is at the heart of the scheduler, as any runnable thread will hit the run queue before the scheduler actually pops it off the queue and runs it. There’s one per capability [rts/Capability.h](https://gitlab.haskell.org/ghc/ghc/blob/master/rts/Capability.h) (in the bad old days, there was a global run queue, but this performed badly for multithreaded processes), and it is implemented as a doubly-linked list `run_queue_hd` and `run_queue_tl`. (It is doubly linked due to ticket #3838). The head and tail pointers mean that the queue is actually a deque: this is important because the scheduler will often have to handle threads that were interrupted in some way, and should let the threads get back on. The links themselves are on the TSOs and modified with `setTSOLink` and `setTSOPrev`, so modifying the queue dirties the TSOs involved. Otherwise, the run queue is exclusively owned by the scheduler. If there are idle capabilities and if we have more than one thread left in our run queue, threads will be pushed to other queues with `schedulePushWork`.
In general, if the thread has exhausted its time slice (`context_switch != 0`), then it goes to the back of the queue, otherwise, it goes on the front and we keep running it.
......@@ -279,7 +279,7 @@ Threads are put in **back** (`appendToRunQueue`) in the case of pre-emption, or
## Sparks and the par operator
Source files: [rts/Sparks.c](https://gitlab.haskell.org/ghc/ghc/tree/master/ghc/rts/Sparks.c), [rts/Sparks.h](/trac/ghc/browser/ghc/rts/Sparks.h).
Source files: [rts/Sparks.c](https://gitlab.haskell.org/ghc/ghc/tree/master/ghc/rts/Sparks.c), [rts/Sparks.h](https://gitlab.haskell.org/ghc/ghc/blob/master/rts/Sparks.h).
The [par](http://www.haskell.org/ghc/docs/latest/html/libraries/base/Control-Parallel.html#v%3Apar) operator is used for annotating computations that could be evaluated in parallel. See also [Parallel Haskell](https://downloads.haskell.org/~ghc/latest/docs/html/users_guide/parallel.html#parallel-haskell) in the GHC User's Guide.
......@@ -291,7 +291,7 @@ Par itself is implemented in terms of the `par#` primop, which the code generato
Par doesn't actually create a new thread immediately; instead it places a pointer to its first argument in the *spark pool*. The spark pool is a circular buffer, when it is full we have the choice of either overwriting the oldest entry or dropping the new entry - currently we drop the new entry (see code for `newSpark`). Each capability has its own spark pool, so this operation can be performed without taking a lock.
So how does the spark turn into a thread? When the scheduler spots that the current capability has no runnable threads, it checks the spark pool, and if there is a valid spark (a spark that points to a THUNK), then the spark is turned into a real thread and placed on the run queue: see `createSparkThread` in [rts/Sparks.c](https://gitlab.haskell.org/ghc/ghc/tree/master/ghc/rts/Sparks.c). Also, the scheduler attempts to share its available sparks with any other idle capabilities: see `schedulePushWork` in [rts/Schedule.c](/trac/ghc/browser/ghc/rts/Schedule.c).
So how does the spark turn into a thread? When the scheduler spots that the current capability has no runnable threads, it checks the spark pool, and if there is a valid spark (a spark that points to a THUNK), then the spark is turned into a real thread and placed on the run queue: see `createSparkThread` in [rts/Sparks.c](https://gitlab.haskell.org/ghc/ghc/tree/master/ghc/rts/Sparks.c). Also, the scheduler attempts to share its available sparks with any other idle capabilities: see `schedulePushWork` in [rts/Schedule.c](https://gitlab.haskell.org/ghc/ghc/blob/master/rts/Schedule.c).
## Affinity and migration
......
......@@ -11,10 +11,10 @@ Source files:
- POSIX signal handling:
- [rts/posix/Signals.h](https://gitlab.haskell.org/ghc/ghc/tree/master/ghc/rts/posix/Signals.h), [rts/posix/Signals.c](/trac/ghc/browser/ghc/rts/posix/Signals.c)
- [rts/posix/Signals.h](https://gitlab.haskell.org/ghc/ghc/tree/master/ghc/rts/posix/Signals.h), [rts/posix/Signals.c](https://gitlab.haskell.org/ghc/ghc/blob/master/rts/posix/Signals.c)
- Windows console events:
- [rts/win32/ConsoleHandler.h](https://gitlab.haskell.org/ghc/ghc/tree/master/ghc/rts/win32/ConsoleHandler.h), [rts/win32/ConsoleHandler.c](/trac/ghc/browser/ghc/rts/win32/ConsoleHandler.c)
- [rts/win32/ConsoleHandler.h](https://gitlab.haskell.org/ghc/ghc/tree/master/ghc/rts/win32/ConsoleHandler.h), [rts/win32/ConsoleHandler.c](https://gitlab.haskell.org/ghc/ghc/blob/master/rts/win32/ConsoleHandler.c)
## Signal handling in the RTS
......@@ -36,7 +36,7 @@ Source files:
- The timer interrupt handler, and starting/stopping the timer:
- [rts/Timer.h](https://gitlab.haskell.org/ghc/ghc/tree/master/ghc/rts/Timer.h), [rts/Timer.c](/trac/ghc/browser/ghc/rts/Timer.c)
- [rts/Timer.h](https://gitlab.haskell.org/ghc/ghc/tree/master/ghc/rts/Timer.h), [rts/Timer.c](https://gitlab.haskell.org/ghc/ghc/blob/master/rts/Timer.c)
- Platform-independent ticker interface, used by the timer:
- [rts/Ticker.h](https://gitlab.haskell.org/ghc/ghc/tree/master/ghc/rts/Ticker.h)
......@@ -66,8 +66,8 @@ When the interrupt signal is received, the default behaviour of the runtime is t
Source files:
- POSIX: [rts/posix/Signals.h](https://gitlab.haskell.org/ghc/ghc/tree/master/ghc/rts/posix/Signals.h), [rts/posix/Signals.c](/trac/ghc/browser/ghc/rts/posix/Signals.c)
- Windows: [rts/win32/ConsoleHandler.h](https://gitlab.haskell.org/ghc/ghc/tree/master/ghc/rts/win32/ConsoleHandler.h), [rts/win32/ConsoleHandler.c](/trac/ghc/browser/ghc/rts/win32/ConsoleHandler.c)
- POSIX: [rts/posix/Signals.h](https://gitlab.haskell.org/ghc/ghc/tree/master/ghc/rts/posix/Signals.h), [rts/posix/Signals.c](https://gitlab.haskell.org/ghc/ghc/blob/master/rts/posix/Signals.c)
- Windows: [rts/win32/ConsoleHandler.h](https://gitlab.haskell.org/ghc/ghc/tree/master/ghc/rts/win32/ConsoleHandler.h), [rts/win32/ConsoleHandler.c](https://gitlab.haskell.org/ghc/ghc/blob/master/rts/win32/ConsoleHandler.c)
A Haskell program can ask to install signal handlers, via the `System.Posix.Signals` API, or `GHC.ConsoleHandler` on Windows. When a signal arrives that has a Haskell handler, it is the job of the runtime to create a new Haskell thread to run the signal handler and place the new thread on the run queue of a suitable [Capability](commentary/rts/scheduler#capabilities).
......
......@@ -171,7 +171,7 @@ The transactional record itself will have an entry for each transactional variab
A transaction starts by initializing a new `TRec` (`stmStartTransaction`) assigning the TSO's `trec` pointer to the new `TRec` then executing the transaction's code.
(See [rts/PrimOps.cmm](https://gitlab.haskell.org/ghc/ghc/tree/master/ghc/rts/PrimOps.cmm)`stg_atomicallyzh` and [rts/STM.c](/trac/ghc/browser/ghc/rts/STM.c)`stmStartTransaction`).
(See [rts/PrimOps.cmm](https://gitlab.haskell.org/ghc/ghc/tree/master/ghc/rts/PrimOps.cmm)`stg_atomicallyzh` and [rts/STM.c](https://gitlab.haskell.org/ghc/ghc/blob/master/rts/STM.c)`stmStartTransaction`).
### Reading
......@@ -221,7 +221,7 @@ With the fine grain lock version when validation, including any read-only phase,
Commit can proceed to increment each locked `TVar`'s `num_updates` field and unlock by writing the new value to the `current_value` field. While these updates happen one-by-one, any attempt to read from this set will spin while the lock is held. Any reads made before the lock was acquired will fail to validate as the number of updates will change.
(See [rts/PrimOps.cmm](https://gitlab.haskell.org/ghc/ghc/tree/master/ghc/rts/PrimOps.cmm)`stg_atomically_frame` and [rts/STM.c](/trac/ghc/browser/ghc/rts/STM.c)`stmCommitTransaction`)
(See [rts/PrimOps.cmm](https://gitlab.haskell.org/ghc/ghc/tree/master/ghc/rts/PrimOps.cmm)`stg_atomically_frame` and [rts/STM.c](https://gitlab.haskell.org/ghc/ghc/blob/master/rts/STM.c)`stmCommitTransaction`)
### Aborting
......@@ -237,7 +237,7 @@ Aborting is simply throwing away changes that are stored in the `TRec`.
An exception in a transaction will only propagate outside of the transaction if the transaction can be validated. If validation fails, the whole transaction will abort and start again from the beginning. Nothing special needs to be done to support the semantics allowing the view *inside* the aborted transaction.
(See [rts/Exception.cmm](https://gitlab.haskell.org/ghc/ghc/tree/master/ghc/rts/Exception.cmm) which calls `stmValidateNestOfTransactions` from [rts/STM.c](/trac/ghc/browser/ghc/rts/STM.c)).
(See [rts/Exception.cmm](https://gitlab.haskell.org/ghc/ghc/tree/master/ghc/rts/Exception.cmm) which calls `stmValidateNestOfTransactions` from [rts/STM.c](https://gitlab.haskell.org/ghc/ghc/blob/master/rts/STM.c)).
---
......@@ -247,7 +247,7 @@ An exception in a transaction will only propagate outside of the transaction if
We will now introduce the blocking feature. To support this we will add a watch queue to each `TVar` where we can place a pointer to a blocked TSO. When a transaction commits we will now wake up the TSOs on watch queues for `TVar`s that are written.
The mechanism for `retry` is similar to exception handling. In the simple case of only supporting blocking and not supporting choice, an encountered retry should validate, and if valid, add the TSO to the watch queue of every accessed `TVar` (see [rts/STM.c](https://gitlab.haskell.org/ghc/ghc/tree/master/ghc/rts/STM.c)`stmWait` and `build_watch_queue_entries_for_trec`). Locks are acquired for all `TVar`s when validating to control access to the watch queues and prevent missing an update to a `TVar` before the thread is sleeping. In particular if validation is successful the locks are held after the return of `stmWait`, through the return to the scheduler, after the thread is safely paused (see [rts/HeapStackCheck.cmm](/trac/ghc/browser/ghc/rts/HeapStackCheck.cmm)`stg_block_stmwait`), and until `stmWaitUnlock` is called. This ensures that no updates to the `TVar`s are made until the TSO is ready to be woken. If validation fails, the `TRec` is discarded and the transaction is started from the beginning. (See [rts/PrimOps.cmm](/trac/ghc/browser/ghc/rts/PrimOps.cmm)`stg_retryzh`)
The mechanism for `retry` is similar to exception handling. In the simple case of only supporting blocking and not supporting choice, an encountered retry should validate, and if valid, add the TSO to the watch queue of every accessed `TVar` (see [rts/STM.c](https://gitlab.haskell.org/ghc/ghc/tree/master/ghc/rts/STM.c)`stmWait` and `build_watch_queue_entries_for_trec`). Locks are acquired for all `TVar`s when validating to control access to the watch queues and prevent missing an update to a `TVar` before the thread is sleeping. In particular if validation is successful the locks are held after the return of `stmWait`, through the return to the scheduler, after the thread is safely paused (see [rts/HeapStackCheck.cmm](https://gitlab.haskell.org/ghc/ghc/blob/master/rts/HeapStackCheck.cmm)`stg_block_stmwait`), and until `stmWaitUnlock` is called. This ensures that no updates to the `TVar`s are made until the TSO is ready to be woken. If validation fails, the `TRec` is discarded and the transaction is started from the beginning. (See [rts/PrimOps.cmm](https://gitlab.haskell.org/ghc/ghc/blob/master/rts/PrimOps.cmm)`stg_retryzh`)
When a transaction is committed, each write that it makes to a `TVar` is preceded by waking up each TSO in the watch queue. Eventually these TSOs will be run, but before restarting the transaction its `TRec` is validated again if valid then nothing has changed that will allow the transaction to proceed with a different result. If invalid, some other transaction has committed and progress may be possible (note there is the additional case that some other transaction is merely holding a lock temporarily, causing validation to fail). The TSO is not removed from the watch queues it is on until the transaction is aborted (at this point we no longer need the `TRec`) and the abort happens after the failure to validate on wakeup. (See [rts/STM.c](https://gitlab.haskell.org/ghc/ghc/tree/master/ghc/rts/STM.c)`stmReWait` and `stmAbortTransaction`)
......@@ -266,7 +266,7 @@ How do we support a "partial abort"? This introduces the need for a nested trans
- **Writing** -- Writes, like reads, now search the parent `TRec`s and the write is stored in the local copy.
- **Retry** -- As described above, we now need to search the stack for a `CATCH_RETRY_FRAME` and if found, aborting the nested transaction and attempting the alternative or propagating the retry instead of immediately working on blocking.
- **Validation** -- If we are validating in the middle of a running transaction we will need to validate the whole nest of transactions.
(See [rts/STM.c](https://gitlab.haskell.org/ghc/ghc/tree/master/ghc/rts/STM.c)`stmValidateNestOfTransactions` and its uses in [rts/Exception.cmm](/trac/ghc/browser/ghc/rts/Exception.cmm) and [rts/Schedule.c](/trac/ghc/browser/ghc/rts/Schedule.c))
(See [rts/STM.c](https://gitlab.haskell.org/ghc/ghc/tree/master/ghc/rts/STM.c)`stmValidateNestOfTransactions` and its uses in [rts/Exception.cmm](https://gitlab.haskell.org/ghc/ghc/blob/master/rts/Exception.cmm) and [rts/Schedule.c](https://gitlab.haskell.org/ghc/ghc/blob/master/rts/Schedule.c))
- **Committing** -- Just as we now have a partial abort, we need a partial commit when we finish a branch of an `orElse`. This commit is done with `stmCommitNestedTransaction` which validates just the inner `TRec` and merges updates back into its parent. Note that an update is distinguished from a read only entry by value. This means that if a nested transaction performs a write that reverts a value this is a change and must still propagate to the parent (see ticket #7493).
- **Aborting** -- There is another subtle issue with how choice and blocking interact. When we block we need to wake up if there is a change to *any* accessed `TVar`. Consider a transaction:
`t = t1 `orElse` t2`
......
# The Block Allocator
Source: [includes/rts/storage/Block.h](https://gitlab.haskell.org/ghc/ghc/tree/master/ghc/includes/rts/storage/Block.h), [rts/sm/BlockAlloc.h](/trac/ghc/browser/ghc/rts/sm/BlockAlloc.h), [rts/sm/BlockAlloc.c](/trac/ghc/browser/ghc/rts/sm/BlockAlloc.c), [includes/rts/storage/MBlock.h](/trac/ghc/browser/ghc/includes/rts/storage/MBlock.h), [rts/sm/MBlock.c](/trac/ghc/browser/ghc/rts/sm/MBlock.c).
Source: [includes/rts/storage/Block.h](https://gitlab.haskell.org/ghc/ghc/tree/master/ghc/includes/rts/storage/Block.h), [rts/sm/BlockAlloc.h](https://gitlab.haskell.org/ghc/ghc/blob/master/rts/sm/BlockAlloc.h), [rts/sm/BlockAlloc.c](https://gitlab.haskell.org/ghc/ghc/blob/master/rts/sm/BlockAlloc.c), [includes/rts/storage/MBlock.h](https://gitlab.haskell.org/ghc/ghc/blob/master/includes/rts/storage/MBlock.h), [rts/sm/MBlock.c](https://gitlab.haskell.org/ghc/ghc/blob/master/rts/sm/MBlock.c).
The block allocator is where the storage manager derives much of its flexibilty. Rather than keep our heap in a single contiguous region of memory, or one contiguous region per generation, we manage linked lists of memory blocks. Managing contiguous regions is difficult, especially when you want to change the size of some of the areas. A block-structured storage arrangement has several advantages:
......@@ -52,7 +52,7 @@ the structure `bdescr` defined in [includes/rts/storage/Block.h](https://gitlab.
The block allocator has a the following structure: