... | ... | @@ -77,7 +77,10 @@ snapshot tarball: |
|
|
from source to .a/.so, including preprocessing (hsc2hs et. al.).
|
|
|
|
|
|
- Use Cabal to preprocess the .cabal file and generate metadata in the
|
|
|
form of Makefile bindings for our build system to use.
|
|
|
form of Makefile bindings for our build system to use. We plan to
|
|
|
generate **only** Makefile bindings this way; we will not generate Makefile
|
|
|
**rules** or shell commands. All Makefile code and shell commands will be
|
|
|
in plain honest-to-goodness Makefiles.
|
|
|
|
|
|
- Use Cabal to generate the InstalledPackageInfo.
|
|
|
|
... | ... | @@ -85,8 +88,8 @@ snapshot tarball: |
|
|
|
|
|
- Use Cabal for Haddocking, and anything else we need to do.
|
|
|
|
|
|
>
|
|
|
> The advantages of this are:
|
|
|
|
|
|
The advantages of this are:
|
|
|
|
|
|
- The critical parts of the build system are under our control, and are
|
|
|
easily modifiable.
|
... | ... | @@ -110,27 +113,11 @@ snapshot tarball: |
|
|
- Compared to the pre-Cabal build system, we're not duplicating the
|
|
|
package metadata or the code that processes it, only the build rules.
|
|
|
|
|
|
## Detailed plan
|
|
|
|
|
|
- Rename cabal-bin.hs to ghc-cabal.hs, and move it into utils/ghc-cabal/
|
|
|
|
|
|
- The version of Cabal used by ghc-cabal.hs does not need to be an
|
|
|
up-to-the-minute bleeding-edge version. It should be stable and vary
|
|
|
slowly. We suck a new version of Cabal into the GHC build system
|
|
|
manually, rather than mirroring the Cabal HEAD.
|
|
|
## Avoiding recursive makefiles
|
|
|
|
|
|
- Rather than installing things in-place all over the build tree, we
|
|
|
will have a single inplace directory at the root of the tree. The
|
|
|
structure inside this directory will match that of the normal install,
|
|
|
which will simplify various things. There are two slight wrinkles:
|
|
|
|
|
|
- The tree will not be complete; for example, the libraries will be
|
|
|
registered in-place in their dist directories
|
|
|
- Rather than inplace/bin/ghc, we will have inplace/bin/ghc-stage\[123\]
|
|
|
Tools like genprimopcode, genapply etc. will probably also go into
|
|
|
inplace/bin, in order to make the makefiles more consistent.
|
|
|
|
|
|
- The build order looks something like:
|
|
|
Currently, with recursive make, this means we jump around between
|
|
|
Makefiles a lot, which isn't good for parallelism in the build. For example, the current build order looks like this:
|
|
|
|
|
|
- With bootstrapping compiler:
|
|
|
|
... | ... | @@ -155,18 +142,39 @@ snapshot tarball: |
|
|
- libraries/\*
|
|
|
- compiler
|
|
|
|
|
|
Currently, with recursive make, this means we jump around between
|
|
|
Makefiles a lot, which isn't good for parallelism in the build.
|
|
|
Instead, we want to move all the logic and dependencies into the root
|
|
|
|
|
|
Instead, following [ Recursive make considered harmful](http://miller.emu.id.au/pmiller/books/rmch/?ref=DDiyet.Com), we propose to move all the logic and dependencies into the root
|
|
|
Makefile (or files that get included into it) so that make sees all of
|
|
|
it together.
|
|
|
it together. Advantages:
|
|
|
|
|
|
- We get to specify dependencies between different parts of the tree much
|
|
|
more easily and precisely. BIG WIN. Have you noticed how often you
|
|
|
need to `make distclean` in a GHC tree to make sure everything is up to
|
|
|
date? (well I rarely do this, because I have the dependencies in my
|
|
|
head and I can rebuild manually, but I imagine this isn't the case for
|
|
|
most people!)
|
|
|
|
|
|
- We get more parallelism, more easily. I'd argue this is a big win too,
|
|
|
right now a validate only uses about 1.3 out of 2 cores.
|
|
|
|
|
|
|
|
|
We don't lose modularity: different parts of the build system are still in
|
|
|
different files, it's just that make sees them all at once. Right now
|
|
|
every make invocation already reads thousands of lines of boilerplate
|
|
|
Makefile code, and we'd be invoking make only once rather than many times.
|
|
|
|
|
|
> >
|
|
|
> > One concern is that make may take a long time thinking if it can see
|
|
|
> > the rules for the whole system, even when only asked to build a single
|
|
|
> > file. We will have to see how well it performs in practice.
|
|
|
|
|
|
- But we still want "make" to work in subdirectories, so for example the
|
|
|
So we will probably have to worry about efficiency. For example, it takes
|
|
|
tens of seconds on Windows for make to discover that compiler/ is up to
|
|
|
date. We don't want that happening every time you rebuild some small part
|
|
|
of the tree, so we plan to cut a few dependencies on purpose. But I'm
|
|
|
hoping it'll be necessary to do this in a few well-defined places only, and
|
|
|
only when building in subdirectories. For example, If you say 'make' in
|
|
|
libraries/base, then we won't try to rebuild the stage1 compiler, we'll
|
|
|
just fail if it does't exist.
|
|
|
|
|
|
|
|
|
We still want "make" to work in subdirectories, so for example the
|
|
|
Makefile (actually GNUmakefile, to avoid colliding with Makefile in
|
|
|
libraries like Cabal) in libraries/base might look like
|
|
|
|
... | ... | @@ -182,18 +190,38 @@ default: dist/build/libbase.a |
|
|
$(MAKE) -C ../.. libraries/base/$@
|
|
|
```
|
|
|
|
|
|
> >
|
|
|
> > (ghc.mk is discussed later). In actual fact, GNUmakefile will want to
|
|
|
> > be more complicated, to handle "make way=v", "make way=p", "make doc",
|
|
|
> > etc. Where possible, the make code will be "include"d in, rather than
|
|
|
> > generated, so as to make it easier to deal with.
|
|
|
|
|
|
> >
|
|
|
> > We need the .NOTPARALLEL or if you say "make foo bar" then the two
|
|
|
> > recursive make calls might both make "quux" (a dependency of foo and
|
|
|
> > bar) at the same time. The main Makefile will be able to do work in
|
|
|
> > parallel when building each of foo and bar, though. The common case,
|
|
|
> > where you only specify 0 or 1 targets, doesn't lose any parallelism.
|
|
|
(ghc.mk is discussed later). In actual fact, GNUmakefile will want to
|
|
|
be more complicated, to handle "make way=v", "make way=p", "make doc",
|
|
|
etc. Where possible, the make code will be "include"d in, rather than
|
|
|
generated, so as to make it easier to deal with.
|
|
|
|
|
|
|
|
|
We need the .NOTPARALLEL or if you say "make foo bar" then the two
|
|
|
recursive make calls might both make "quux" (a dependency of foo and
|
|
|
bar) at the same time. The main Makefile will be able to do work in
|
|
|
parallel when building each of foo and bar, though. The common case,
|
|
|
where you only specify 0 or 1 targets, doesn't lose any parallelism.
|
|
|
|
|
|
## Detailed plan
|
|
|
|
|
|
- Rename cabal-bin.hs to ghc-cabal.hs, and move it into utils/ghc-cabal/
|
|
|
|
|
|
- The version of Cabal used by ghc-cabal.hs does not need to be an
|
|
|
up-to-the-minute bleeding-edge version. It should be stable and vary
|
|
|
slowly. We suck a new version of Cabal into the GHC build system
|
|
|
manually, rather than mirroring the Cabal HEAD.
|
|
|
|
|
|
- Rather than installing things in-place all over the build tree, we
|
|
|
will have a single inplace directory at the root of the tree. The
|
|
|
structure inside this directory will match that of the normal install,
|
|
|
which will simplify various things. There are two slight wrinkles:
|
|
|
|
|
|
- The tree will not be complete; for example, the libraries will be
|
|
|
registered in-place in their dist directories
|
|
|
- Rather than inplace/bin/ghc, we will have inplace/bin/ghc-stage\[123\]
|
|
|
Tools like genprimopcode, genapply etc. will probably also go into
|
|
|
inplace/bin, in order to make the makefiles more consistent.
|
|
|
|
|
|
- In e.g. utils/ghc-pkg, the default target will be
|
|
|
|
... | ... | |