nix-local-build.rst 10.4 KB
Newer Older
Leonid Onokhov's avatar
Leonid Onokhov committed
1
.. highlight:: console
2 3 4 5 6

Quickstart
==========

Suppose that you are in a directory containing a single Cabal package
7
which you wish to build (if you haven't set up a package yet check
8
out `developing packages <developing-packages.html>`__ for
9
instructions). You can configure and build it using Nix-style
10 11 12 13
local builds with this command (configuring is not necessary):

::

14
    $ cabal v2-build
15 16 17 18 19

To open a GHCi shell with this package, use this command:

::

20
    $ cabal v2-repl
21

Francesco Gazzetta's avatar
Francesco Gazzetta committed
22 23 24 25
To run an executable defined in this package, use this command:

::

26
    $ cabal v2-run <executable name> [executable args]
Francesco Gazzetta's avatar
Francesco Gazzetta committed
27

28 29 30 31 32 33 34 35 36 37 38
Developing multiple packages
----------------------------

Many Cabal projects involve multiple packages which need to be built
together. To build multiple Cabal packages, you need to first create a
``cabal.project`` file which declares where all the local package
directories live. For example, in the Cabal repository, there is a root
directory with a folder per package, e.g., the folders ``Cabal`` and
``cabal-install``. The ``cabal.project`` file specifies each folder as
part of the project:

Leonid Onokhov's avatar
Leonid Onokhov committed
39
.. code-block:: cabal
40 41 42 43 44 45 46 47 48 49

    packages: Cabal/
              cabal-install/

The expectation is that a ``cabal.project`` is checked into your source
control, to be used by all developers of a project. If you need to make
local changes, they can be placed in ``cabal.project.local`` (which
should not be checked in.)

Then, to build every component of every package, from the top-level
50
directory, run the command: (using cabal-install-2.0 or greater.)
51 52 53

::

54
    $ cabal v2-build all
55

56
To build a specific package, you can either run ``v2-build`` from the
57 58 59 60
directory of the package in question:

::

Leonid Onokhov's avatar
Leonid Onokhov committed
61
    $ cd cabal-install
62
    $ cabal v2-build
63 64

or you can pass the name of the package as an argument to
65
``cabal v2-build`` (this works in any subdirectory of the project):
66 67 68

::

69
    $ cabal v2-build cabal-install
70 71 72 73 74 75

You can also specify a specific component of the package to build. For
example, to build a test suite named ``package-tests``, use the command:

::

76
    $ cabal v2-build package-tests
77 78 79 80 81 82 83

Targets can be qualified with package names. So to request
``package-tests`` *from* the ``Cabal`` package, use
``Cabal:package-tests``.

Unlike sandboxes, there is no need to setup a sandbox or ``add-source``
projects; just check in ``cabal.project`` to your repository and
84
``v2-build`` will just work.
85

86 87 88 89 90 91 92 93 94 95 96
Cookbook
========

How can I profile my library/application?
-----------------------------------------

Create or edit your ``cabal.project.local``, adding the following
line::

    profiling: True

97
Now, ``cabal v2-build`` will automatically build all libraries and
98 99 100 101 102 103
executables with profiling.  You can fine-tune the profiling settings
for each package using :cfg-field:`profiling-detail`::

    package p
        profiling-detail: toplevel-functions

104
Alternately, you can call ``cabal v2-build --enable-profiling`` to
105 106
temporarily build with profiling.

107 108 109 110 111 112 113 114 115
How it works
============

Local versus external packages
------------------------------

One of the primary innovations of Nix-style local builds is the
distinction between local packages, which users edit and recompile and
must be built per-project, versus external packages, which can be cached
116
across projects. To be more precise:
117 118 119 120

1. A **local package** is one that is listed explicitly in the
   ``packages``, ``optional-packages`` or ``extra-packages`` field of a
   project. Usually, these refer to packages whose source code lives
Jan Hrček's avatar
Jan Hrček committed
121 122 123
   directly in a folder in your project. But you can list an
   arbitrary Hackage package in `extra-packages <cabal-project.html#cfg-field-extra-packages>`
   to force it to be treated as local.
124 125 126 127 128 129 130

Local packages, as well as the external packages (below) which depend on
them, are built **inplace**, meaning that they are always built
specifically for the project and are not installed globally. Inplace
packages are not cached and not given unique hashes, which makes them
suitable for packages which you want to edit and recompile.

131
2. An **external package** is any package which is not listed in the
132 133 134 135 136
   ``packages`` field. The source code for external packages is usually
   retrieved from Hackage.

When an external package does not depend on an inplace package, it can
be built and installed to a **global** store, which can be shared across
Jan Hrček's avatar
Jan Hrček committed
137 138
projects. These build products are identified by a hash based on all of
the inputs which influence the compilation of a package (flags,
139 140 141 142 143
dependency selection, etc.). Just as in Nix, these hashes uniquely
identify the result of a build; if we compute this identifier and we
find that we already have this ID built, we can just use the already
built version.

144 145
The global package store is ``~/.cabal/store`` (configurable via
global `store-dir` option); if you need to clear your store for
Alex Biehl's avatar
Alex Biehl committed
146
whatever reason (e.g., to reclaim disk space or because the global
147
store is corrupted), deleting this directory is safe (``v2-build``
Alex Biehl's avatar
Alex Biehl committed
148
will just rebuild everything it needs on its next invocation).
149 150

This split motivates some of the UI choices for Nix-style local build
151
commands. For example, flags passed to ``cabal v2-build`` are only
152
applied to *local* packages, so that adding a flag to
153
``cabal v2-build`` doesn't necessitate a rebuild of *every* transitive
154 155
dependency in the global package store.

156
In cabal-install 2.0 and above, Nix-style local builds also take advantage of a
157 158 159 160 161 162 163 164 165 166 167
new Cabal library feature, `per-component
builds <https://github.com/ezyang/ghc-proposals/blob/master/proposals/0000-componentized-cabal.rst>`__,
where each component of a package is configured and built separately.
This can massively speed up rebuilds of packages with lots of components
(e.g., a package that defines multiple executables), as only one
executable needs to be rebuilt. Packages that use Custom setup scripts
are not currently built on a per-component basis.

Where are my build products?
----------------------------

168
A major deficiency in the current implementation of v2-build is that
169 170
there is no programmatic way to access the location of build products.
The location of the build products is intended to be an internal
171
implementation detail of v2-build, but we also understand that many
172 173
unimplemented features can only be reasonably worked around by
accessing build products directly.
174 175 176 177 178 179 180 181 182

The location where build products can be found varies depending on the
version of cabal-install:

-  In cabal-install-1.24, the dist directory for a package ``p-0.1`` is
   stored in ``dist-newstyle/build/p-0.1``. For example, if you built an
   executable or test suite named ``pexe``, it would be located at
   ``dist-newstyle/build/p-0.1/build/pexe/pexe``.

183
-  In cabal-install-2.0, the dist directory for a package ``p-0.1``
184 185 186 187 188 189 190 191 192 193
   defining a library built with GHC 8.0.1 on 64-bit Linux is
   ``dist-newstyle/build/x86_64-linux/ghc-8.0.1/p-0.1``. When
   per-component builds are enabled (any non-Custom package), a
   subcomponent like an executable or test suite named ``pexe`` will be
   stored at
   ``dist-newstyle/build/x86_64-linux/ghc-8.0.1/p-0.1/c/pexe``; thus,
   the full path of the executable is
   ``dist-newstyle/build/x86_64-linux/ghc-8.0.1/p-0.1/c/pexe/build/pexe/pexe``
   (you can see why we want this to be an implementation detail!)

Jan Hrček's avatar
Jan Hrček committed
194
-  In cabal-install-2.2 and above, the ``/c/`` part of the above path
195 196 197 198 199 200 201 202 203 204
   is replaced with one of ``/l/``, ``/x/``, ``/f/``, ``/t/``, or
   ``/b/``, depending on the type of component (sublibrary,
   executable, foreign library, test suite, or benchmark
   respectively). So the full path to an executable named ``pexe``
   compiled with GHC 8.0.1 on a 64-bit Linux is now
   ``dist-newstyle/build/x86_64-linux/ghc-8.0.1/p-0.1/x/pexe/build/pexe/pexe``;
   for a benchmark named ``pbench`` it now is
   ``dist-newstyle/build/x86_64-linux/ghc-8.0.1/p-0.1/b/pbench/build/pbench/pbench``;


205
The paths are a bit longer in 2.0 and above but the benefit is that you can
206 207 208 209 210 211 212 213
transparently have multiple builds with different versions of GHC. We
plan to add the ability to create aliases for certain build
configurations, and more convenient paths to access particularly useful
build products like executables.

Caching
-------

Jan Hrček's avatar
Jan Hrček committed
214
Nix-style local builds sport a robust caching system which helps to reduce
215 216 217 218
the time it takes to execute a rebuild cycle. While the details of how
``cabal-install`` does caching are an implementation detail and may
change in the future, knowing what gets cached is helpful for
understanding the performance characteristics of invocations to
219
``v2-build``. The cached intermediate results are stored in
220 221 222 223 224 225 226 227 228 229 230 231 232 233 234
``dist-newstyle/cache``; this folder can be safely deleted to clear the
cache.

The following intermediate results are cached in the following files in
this folder (the most important two are first):

``solver-plan`` (binary)
    The result of calling the dependency solver, assuming that the
    Hackage index, local ``cabal.project`` file, and local ``cabal``
    files are unmodified. (Notably, we do NOT have to dependency solve
    again if new build products are stored in the global store; the
    invocation of the dependency solver is independent of what is
    already available in the store.)
``source-hashes`` (binary)
    The hashes of all local source files. When all local source files of
235
    a local package are unchanged, ``cabal v2-build`` will skip
236 237
    invoking ``setup build`` entirely (saving us from a possibly
    expensive call to ``ghc --make``). The full list of source files
Jan Hrček's avatar
Jan Hrček committed
238 239 240 241
    participating in compilation is determined using
    ``cabal sdist --list-only``. Thus if you do not list all your
    source files in a Cabal file, Cabal may fail to recompile when you
    edit them.
242 243 244 245 246 247 248 249
``config`` (same format as ``cabal.project``)
    The full project configuration, merged from ``cabal.project`` (and
    friends) as well as the command line arguments.
``compiler`` (binary)
    The configuration of the compiler being used to build the project.
``improved-plan`` (binary)
    Like ``solver-plan``, but with all non-inplace packages improved
    into pre-existing copies from the store.
250 251 252 253 254 255 256 257 258 259 260 261
``plan.json`` (JSON)
    A JSON serialization of the computed install plan intended
    for integrating ``cabal`` with external tooling.
    The `cabal-plan <http://hackage.haskell.org/package/cabal-plan>`__
    package provides a library for parsing ``plan.json`` files into a
    Haskell data structure as well as an example tool showing possible
    applications.

    .. todo::

        Document JSON schema (including version history of schema)

262 263

Note that every package also has a local cache managed by the Cabal
Jan Hrček's avatar
Jan Hrček committed
264
build system, e.g., in ``$distdir/cache``.