... | ... | @@ -5,7 +5,7 @@ |
|
|
|
|
|
When GHC is compiling a module, it tries to determine early on whether
|
|
|
|
|
|
- The object file (or byte-code in the case of GHCi) and [interface file](commentary/compiler/iface-files) exist from a previous compilation
|
|
|
- The results of compilations exist from a previous compilation. This includes [interface files](commentary/compiler/iface-files), hie files, object files and dynamic object files. T
|
|
|
- Recompilation is sure to produce exactly the same results, so it
|
|
|
is not necessary.
|
|
|
|
... | ... | @@ -15,6 +15,8 @@ existing object code and interface are still valid. In GHCi and |
|
|
`--make`, we must generate the `ModDetails` from the `ModIface`, but
|
|
|
this is easily done by calling `MkIface.typecheckIface`.
|
|
|
|
|
|
The recompilation check is **entirely** implemented in `checkRecompile`. This is the **only** place in the compiler where we make any decision about recompilation.
|
|
|
|
|
|
## Example
|
|
|
|
|
|
|
... | ... | @@ -79,80 +81,7 @@ depends directly or indirectly on something that changed. In GHCi and |
|
|
order, and decides whether it needs to be recompiled, or whether the
|
|
|
existing object code and interface will do.
|
|
|
|
|
|
### `make`
|
|
|
|
|
|
`make` works by checking the timestamps on dependencies and
|
|
|
recompiling things when the dependencies are newer. Dependency lists
|
|
|
for `make` look like this (generated by `ghc -M`):
|
|
|
|
|
|
```wiki
|
|
|
# DO NOT DELETE: Beginning of Haskell dependencies
|
|
|
D.o : D.hs
|
|
|
B.o : B.hs
|
|
|
B.o : D.hi
|
|
|
C.o : C.hs
|
|
|
C.o : D.hi
|
|
|
A.o : A.hs
|
|
|
A.o : C.hi
|
|
|
A.o : B.hi
|
|
|
# DO NOT DELETE: End of Haskell dependencies
|
|
|
```
|
|
|
|
|
|
|
|
|
Only the `.hi` files of the *direct imports* of a module are listed.
|
|
|
For example, `A.o` depends on `C.hi` and `B.hi`, but not `D.hi`.
|
|
|
Nevertheless, if D is modified, we might need to recompile A. How
|
|
|
does this happen?
|
|
|
|
|
|
- first, make will recompile D because its source file has changed,
|
|
|
generating a new `D.o` and `D.hi`.
|
|
|
|
|
|
- If after recompiling D, we notice that its interface is the same
|
|
|
as before, there is no need to modify the `.hi` file. If the `.hi`
|
|
|
file is not modified by the compilation, then `make` will notice
|
|
|
and not recompile `B` or `C`, or indeed `A`. This is an important
|
|
|
optimisation.
|
|
|
|
|
|
- Suppose the change to `D` did cause a change in the interface
|
|
|
(e.g. the type of `f` changed). Now, `make` will recompile both
|
|
|
`B` and `C`. Suppose that the interfaces to `B` and `C`
|
|
|
remain the same: B's interface says only that it re-exports `D.f`,
|
|
|
so the fact that `f` has a new type does not affect `B`'s
|
|
|
interface.
|
|
|
|
|
|
- Now, `A`'s dependencies are unchanged, so `A` will not be
|
|
|
recompiled. But this is wrong: `A` might depend on something from
|
|
|
`D` that was re-exported via `B` or `C`, and therefore need
|
|
|
recompiling.
|
|
|
|
|
|
|
|
|
To ensure that `A` is recompiled, we therefore have two options:
|
|
|
|
|
|
1. arrange that make knows about the dependency of A on D.
|
|
|
|
|
|
1. arrange to touch `B.hi` and `C.hi` even if they haven't changed.
|
|
|
|
|
|
|
|
|
GHC currently does (2), more about that in a minute.
|
|
|
|
|
|
|
|
|
Why not do (1)? Well, then *every* time `D.hi` changed, GHC would be
|
|
|
invoked on `A` again. But `A` doesn't depend directly on `D`: it
|
|
|
imports `B`, and it might be therefore be insensitive to changes in `D`.
|
|
|
By telling make only about direct dependencies, we gain the ability to
|
|
|
avoid recompiling modules further up the dependency graph, by not touching
|
|
|
interface files when they don't change.
|
|
|
|
|
|
|
|
|
Back to (2). In addition to correctness (recompile when necessary), we also want to
|
|
|
avoid unnecessary recompilation as far as possible.
|
|
|
Make only knows about very coarse-grained dependencies. For example,
|
|
|
it doesn't know that changing the type of `D.f` can have no effect on
|
|
|
`C`, so `C` does not in fact need to be recompiled, because to do so
|
|
|
would generate exactly the same `.o` and `.hi` files as last time.
|
|
|
GHC does have enough information to figure this out, so when GHC is
|
|
|
asked to recompile a module it invokes the *recompilation checker*
|
|
|
to determine whether recompilation can be avoided in this case.
|
|
|
|
|
|
## How does it work?
|
|
|
|
... | ... | @@ -173,6 +102,10 @@ An [interface file](commentary/compiler/iface-files) contains: |
|
|
interface didn't change at all, then we don't want to touch the
|
|
|
on-disk version because that would cause `make` to perform more
|
|
|
compilations.
|
|
|
- The *source hash*, which is a hash of the input file. This is used to determine
|
|
|
whether a file has been modified or not. In the past we used to use the modification time
|
|
|
of files but using the source hash is more robust to poor resolution timers and modern build
|
|
|
systems.
|
|
|
- The *ABI hash*, which depends on everything that the module
|
|
|
exposes about its implementation: think of this as a hash of
|
|
|
*export-list hash* and *decls*.
|
... | ... | @@ -180,9 +113,9 @@ An [interface file](commentary/compiler/iface-files) contains: |
|
|
|
|
|
- The export list itself. The export-list hash only depends on the *names* of the exports for the modules. The *types* of these exports are ignored in calculating the hash. Only a change of name or removal or addition of an export will change the hash. Not a type change of definition change.
|
|
|
- the *orphan hash*, which depends on all the orphan instances/rules in the, and the orphan hashes of all orphan modules below this module in the dependency tree (see [Orphans](commentary/compiler/recompilation-avoidance#orphans)).
|
|
|
- the package dependencies (see [Package Version Changes](commentary/compiler/recompilation-avoidance#package-version-changes)).
|
|
|
- the **direct** package dependencies (see [Package Version Changes](commentary/compiler/recompilation-avoidance#package-version-changes)).
|
|
|
- *exports*: what the module exports
|
|
|
- *dependencies*: modules and packages that this module depends on
|
|
|
- *dependencies*: modules and packages that this module **directly** depends on
|
|
|
- *usages*: what specific entities the module depends on
|
|
|
- *decls*: what the module defines
|
|
|
- various other stuff, but the above are the important bits
|
... | ... | @@ -610,3 +543,78 @@ For recompilation avoidance to be really effective, we need to ensure that finge |
|
|
In GHC 6.12 we changed this so that compiler-generated bindings are given names of the form `f_x`, where `f` is the name of the exported Id that refers to the binding. If there are multiple `f_x`s, then they are disambiguated with an integer suffix, but the numbers are assigned deterministically, by traversing the definition of `f` in depth-first left-to-right order to find references. See `TidyPgm.chooseExternalIds`.
|
|
|
|
|
|
- There are still some cases where an interface can change without changing the source code. The ones we know about are listed in #4012
|
|
|
|
|
|
### `make`
|
|
|
|
|
|
`make` works by checking the timestamps on dependencies and
|
|
|
recompiling things when the dependencies are newer. Dependency lists
|
|
|
for `make` look like this (generated by `ghc -M`):
|
|
|
|
|
|
```wiki
|
|
|
# DO NOT DELETE: Beginning of Haskell dependencies
|
|
|
D.o : D.hs
|
|
|
B.o : B.hs
|
|
|
B.o : D.hi
|
|
|
C.o : C.hs
|
|
|
C.o : D.hi
|
|
|
A.o : A.hs
|
|
|
A.o : C.hi
|
|
|
A.o : B.hi
|
|
|
# DO NOT DELETE: End of Haskell dependencies
|
|
|
```
|
|
|
|
|
|
|
|
|
Only the `.hi` files of the *direct imports* of a module are listed.
|
|
|
For example, `A.o` depends on `C.hi` and `B.hi`, but not `D.hi`.
|
|
|
Nevertheless, if D is modified, we might need to recompile A. How
|
|
|
does this happen?
|
|
|
|
|
|
- first, make will recompile D because its source file has changed,
|
|
|
generating a new `D.o` and `D.hi`.
|
|
|
|
|
|
- If after recompiling D, we notice that its interface is the same
|
|
|
as before, there is no need to modify the `.hi` file. If the `.hi`
|
|
|
file is not modified by the compilation, then `make` will notice
|
|
|
and not recompile `B` or `C`, or indeed `A`. This is an important
|
|
|
optimisation.
|
|
|
|
|
|
- Suppose the change to `D` did cause a change in the interface
|
|
|
(e.g. the type of `f` changed). Now, `make` will recompile both
|
|
|
`B` and `C`. Suppose that the interfaces to `B` and `C`
|
|
|
remain the same: B's interface says only that it re-exports `D.f`,
|
|
|
so the fact that `f` has a new type does not affect `B`'s
|
|
|
interface.
|
|
|
|
|
|
- Now, `A`'s dependencies are unchanged, so `A` will not be
|
|
|
recompiled. But this is wrong: `A` might depend on something from
|
|
|
`D` that was re-exported via `B` or `C`, and therefore need
|
|
|
recompiling.
|
|
|
|
|
|
|
|
|
To ensure that `A` is recompiled, we therefore have two options:
|
|
|
|
|
|
1. arrange that make knows about the dependency of A on D.
|
|
|
|
|
|
1. arrange to touch `B.hi` and `C.hi` even if they haven't changed.
|
|
|
|
|
|
|
|
|
GHC currently does (2), more about that in a minute.
|
|
|
|
|
|
|
|
|
Why not do (1)? Well, then *every* time `D.hi` changed, GHC would be
|
|
|
invoked on `A` again. But `A` doesn't depend directly on `D`: it
|
|
|
imports `B`, and it might be therefore be insensitive to changes in `D`.
|
|
|
By telling make only about direct dependencies, we gain the ability to
|
|
|
avoid recompiling modules further up the dependency graph, by not touching
|
|
|
interface files when they don't change.
|
|
|
|
|
|
|
|
|
Back to (2). In addition to correctness (recompile when necessary), we also want to
|
|
|
avoid unnecessary recompilation as far as possible.
|
|
|
Make only knows about very coarse-grained dependencies. For example,
|
|
|
it doesn't know that changing the type of `D.f` can have no effect on
|
|
|
`C`, so `C` does not in fact need to be recompiled, because to do so
|
|
|
would generate exactly the same `.o` and `.hi` files as last time.
|
|
|
GHC does have enough information to figure this out, so when GHC is
|
|
|
asked to recompile a module it invokes the *recompilation checker*
|
|
|
to determine whether recompilation can be avoided in this case. |