nix-local-build.rst 77 KB
Newer Older
Leonid Onokhov's avatar
Leonid Onokhov committed
1
.. highlight:: console
2 3 4 5 6

Quickstart
==========

Suppose that you are in a directory containing a single Cabal package
7
which you wish to build (if you haven't set up a package yet check
8
out `developing packages <developing-packages.html>`__ for
9
instructions). You can configure and build it using Nix-style
10 11 12 13
local builds with this command (configuring is not necessary):

::

14
    $ cabal v2-build
15 16 17 18 19

To open a GHCi shell with this package, use this command:

::

20
    $ cabal v2-repl
21

Francesco Gazzetta's avatar
Francesco Gazzetta committed
22 23 24 25
To run an executable defined in this package, use this command:

::

26
    $ cabal v2-run <executable name> [executable args]
Francesco Gazzetta's avatar
Francesco Gazzetta committed
27

28 29 30 31 32 33 34 35 36 37 38
Developing multiple packages
----------------------------

Many Cabal projects involve multiple packages which need to be built
together. To build multiple Cabal packages, you need to first create a
``cabal.project`` file which declares where all the local package
directories live. For example, in the Cabal repository, there is a root
directory with a folder per package, e.g., the folders ``Cabal`` and
``cabal-install``. The ``cabal.project`` file specifies each folder as
part of the project:

Leonid Onokhov's avatar
Leonid Onokhov committed
39
.. code-block:: cabal
40 41 42 43 44 45 46 47 48 49

    packages: Cabal/
              cabal-install/

The expectation is that a ``cabal.project`` is checked into your source
control, to be used by all developers of a project. If you need to make
local changes, they can be placed in ``cabal.project.local`` (which
should not be checked in.)

Then, to build every component of every package, from the top-level
50
directory, run the command: (using cabal-install-2.0 or greater.)
51 52 53

::

54
    $ cabal v2-build
55

56
To build a specific package, you can either run ``v2-build`` from the
57 58 59 60
directory of the package in question:

::

Leonid Onokhov's avatar
Leonid Onokhov committed
61
    $ cd cabal-install
62
    $ cabal v2-build
63 64

or you can pass the name of the package as an argument to
65
``cabal v2-build`` (this works in any subdirectory of the project):
66 67 68

::

69
    $ cabal v2-build cabal-install
70 71 72 73 74 75

You can also specify a specific component of the package to build. For
example, to build a test suite named ``package-tests``, use the command:

::

76
    $ cabal v2-build package-tests
77 78 79 80 81 82 83

Targets can be qualified with package names. So to request
``package-tests`` *from* the ``Cabal`` package, use
``Cabal:package-tests``.

Unlike sandboxes, there is no need to setup a sandbox or ``add-source``
projects; just check in ``cabal.project`` to your repository and
84
``v2-build`` will just work.
85

86 87 88 89 90 91 92 93 94 95 96
Cookbook
========

How can I profile my library/application?
-----------------------------------------

Create or edit your ``cabal.project.local``, adding the following
line::

    profiling: True

97
Now, ``cabal v2-build`` will automatically build all libraries and
98 99 100 101 102 103
executables with profiling.  You can fine-tune the profiling settings
for each package using :cfg-field:`profiling-detail`::

    package p
        profiling-detail: toplevel-functions

104
Alternately, you can call ``cabal v2-build --enable-profiling`` to
105 106
temporarily build with profiling.

107 108 109 110 111 112 113 114 115
How it works
============

Local versus external packages
------------------------------

One of the primary innovations of Nix-style local builds is the
distinction between local packages, which users edit and recompile and
must be built per-project, versus external packages, which can be cached
116
across projects. To be more precise:
117 118 119 120 121

1. A **local package** is one that is listed explicitly in the
   ``packages``, ``optional-packages`` or ``extra-packages`` field of a
   project. Usually, these refer to packages whose source code lives
   directly in a folder in your project (although, you can list an
122
   arbitrary Hackage package in ``extra-packages`` to force it to be
123 124 125 126 127 128 129 130
   treated as local).

Local packages, as well as the external packages (below) which depend on
them, are built **inplace**, meaning that they are always built
specifically for the project and are not installed globally. Inplace
packages are not cached and not given unique hashes, which makes them
suitable for packages which you want to edit and recompile.

131
2. An **external package** is any package which is not listed in the
132 133 134 135 136 137 138 139 140 141 142 143
   ``packages`` field. The source code for external packages is usually
   retrieved from Hackage.

When an external package does not depend on an inplace package, it can
be built and installed to a **global** store, which can be shared across
projects. These build products are identified by a hash that over all of
the inputs which would influence the compilation of a package (flags,
dependency selection, etc.). Just as in Nix, these hashes uniquely
identify the result of a build; if we compute this identifier and we
find that we already have this ID built, we can just use the already
built version.

144 145
The global package store is ``~/.cabal/store`` (configurable via
global `store-dir` option); if you need to clear your store for
Alex Biehl's avatar
Alex Biehl committed
146
whatever reason (e.g., to reclaim disk space or because the global
147
store is corrupted), deleting this directory is safe (``v2-build``
Alex Biehl's avatar
Alex Biehl committed
148
will just rebuild everything it needs on its next invocation).
149 150

This split motivates some of the UI choices for Nix-style local build
151
commands. For example, flags passed to ``cabal v2-build`` are only
152
applied to *local* packages, so that adding a flag to
153
``cabal v2-build`` doesn't necessitate a rebuild of *every* transitive
154 155
dependency in the global package store.

156
In cabal-install 2.0 and above, Nix-style local builds also take advantage of a
157 158 159 160 161 162 163 164 165 166 167
new Cabal library feature, `per-component
builds <https://github.com/ezyang/ghc-proposals/blob/master/proposals/0000-componentized-cabal.rst>`__,
where each component of a package is configured and built separately.
This can massively speed up rebuilds of packages with lots of components
(e.g., a package that defines multiple executables), as only one
executable needs to be rebuilt. Packages that use Custom setup scripts
are not currently built on a per-component basis.

Where are my build products?
----------------------------

168
A major deficiency in the current implementation of v2-build is that
169 170
there is no programmatic way to access the location of build products.
The location of the build products is intended to be an internal
171
implementation detail of v2-build, but we also understand that many
172 173
unimplemented features can only be reasonably worked around by
accessing build products directly.
174 175 176 177 178 179 180 181 182

The location where build products can be found varies depending on the
version of cabal-install:

-  In cabal-install-1.24, the dist directory for a package ``p-0.1`` is
   stored in ``dist-newstyle/build/p-0.1``. For example, if you built an
   executable or test suite named ``pexe``, it would be located at
   ``dist-newstyle/build/p-0.1/build/pexe/pexe``.

183
-  In cabal-install-2.0, the dist directory for a package ``p-0.1``
184 185 186 187 188 189 190 191 192 193
   defining a library built with GHC 8.0.1 on 64-bit Linux is
   ``dist-newstyle/build/x86_64-linux/ghc-8.0.1/p-0.1``. When
   per-component builds are enabled (any non-Custom package), a
   subcomponent like an executable or test suite named ``pexe`` will be
   stored at
   ``dist-newstyle/build/x86_64-linux/ghc-8.0.1/p-0.1/c/pexe``; thus,
   the full path of the executable is
   ``dist-newstyle/build/x86_64-linux/ghc-8.0.1/p-0.1/c/pexe/build/pexe/pexe``
   (you can see why we want this to be an implementation detail!)

194 195 196 197 198 199 200 201 202 203 204
- In cabal-install-2.2 and above, the ``/c/`` part of the above path
   is replaced with one of ``/l/``, ``/x/``, ``/f/``, ``/t/``, or
   ``/b/``, depending on the type of component (sublibrary,
   executable, foreign library, test suite, or benchmark
   respectively). So the full path to an executable named ``pexe``
   compiled with GHC 8.0.1 on a 64-bit Linux is now
   ``dist-newstyle/build/x86_64-linux/ghc-8.0.1/p-0.1/x/pexe/build/pexe/pexe``;
   for a benchmark named ``pbench`` it now is
   ``dist-newstyle/build/x86_64-linux/ghc-8.0.1/p-0.1/b/pbench/build/pbench/pbench``;


205
The paths are a bit longer in 2.0 and above but the benefit is that you can
206 207 208 209 210 211 212 213 214 215 216 217 218
transparently have multiple builds with different versions of GHC. We
plan to add the ability to create aliases for certain build
configurations, and more convenient paths to access particularly useful
build products like executables.

Caching
-------

Nix-style local builds sport a robust caching system which help reduce
the time it takes to execute a rebuild cycle. While the details of how
``cabal-install`` does caching are an implementation detail and may
change in the future, knowing what gets cached is helpful for
understanding the performance characteristics of invocations to
219
``v2-build``. The cached intermediate results are stored in
220 221 222 223 224 225 226 227 228 229 230 231 232 233 234
``dist-newstyle/cache``; this folder can be safely deleted to clear the
cache.

The following intermediate results are cached in the following files in
this folder (the most important two are first):

``solver-plan`` (binary)
    The result of calling the dependency solver, assuming that the
    Hackage index, local ``cabal.project`` file, and local ``cabal``
    files are unmodified. (Notably, we do NOT have to dependency solve
    again if new build products are stored in the global store; the
    invocation of the dependency solver is independent of what is
    already available in the store.)
``source-hashes`` (binary)
    The hashes of all local source files. When all local source files of
235
    a local package are unchanged, ``cabal v2-build`` will skip
236 237 238 239 240 241 242 243 244 245 246 247 248 249
    invoking ``setup build`` entirely (saving us from a possibly
    expensive call to ``ghc --make``). The full list of source files
    participating in compilation are determined using
    ``setup sdist --list-sources`` (thus, if you do not list all your
    source files in a Cabal file, you may fail to recompile when you
    edit them.)
``config`` (same format as ``cabal.project``)
    The full project configuration, merged from ``cabal.project`` (and
    friends) as well as the command line arguments.
``compiler`` (binary)
    The configuration of the compiler being used to build the project.
``improved-plan`` (binary)
    Like ``solver-plan``, but with all non-inplace packages improved
    into pre-existing copies from the store.
250 251 252 253 254 255 256 257 258 259 260 261
``plan.json`` (JSON)
    A JSON serialization of the computed install plan intended
    for integrating ``cabal`` with external tooling.
    The `cabal-plan <http://hackage.haskell.org/package/cabal-plan>`__
    package provides a library for parsing ``plan.json`` files into a
    Haskell data structure as well as an example tool showing possible
    applications.

    .. todo::

        Document JSON schema (including version history of schema)

262 263 264 265

Note that every package also has a local cache managed by the Cabal
build system, e.g., in ``$distdir/cache``.

266 267 268 269 270 271
There is another useful file in ``dist-newstyle/cache``,
``plan.json``, which is a JSON serialization of the computed install
plan and is intended for integrating with external tooling.



272 273 274 275 276 277 278

Commands
========

We now give an in-depth description of all the commands, describing the
arguments and flags they accept.

279
cabal v2-configure
280 281
-------------------

282
``cabal v2-configure`` takes a set of arguments and writes a
283
``cabal.project.local`` file based on the flags passed to this command.
284 285 286
``cabal v2-configure FLAGS; cabal new-build`` is roughly equivalent to
``cabal v2-build FLAGS``, except that with ``new-configure`` the flags
are persisted to all subsequent calls to ``v2-build``.
287

288
``cabal v2-configure`` is intended to be a convenient way to write out
289
a ``cabal.project.local`` for simple configurations; e.g.,
290 291
``cabal v2-configure -w ghc-7.8`` would ensure that all subsequent
builds with ``cabal v2-build`` are performed with the compiler
292 293 294 295
``ghc-7.8``. For more complex configuration, we recommend writing the
``cabal.project.local`` file directly (or placing it in
``cabal.project``!)

296
``cabal v2-configure`` inherits options from ``Cabal``. semantics:
297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316

-  Any flag accepted by ``./Setup configure``.

-  Any flag accepted by ``cabal configure`` beyond
   ``./Setup configure``, namely ``--cabal-lib-version``,
   ``--constraint``, ``--preference`` and ``--solver.``

-  Any flag accepted by ``cabal install`` beyond ``./Setup configure``.

-  Any flag accepted by ``./Setup haddock``.

The options of all of these flags apply only to *local* packages in a
project; this behavior is different than that of ``cabal install``,
which applies flags to every package that would be built. The motivation
for this is to avoid an innocuous addition to the flags of a package
resulting in a rebuild of every package in the store (which might need
to happen if a flag actually applied to every transitive dependency). To
apply options to an external package, use a ``package`` stanza in a
``cabal.project`` file.

317
cabal v2-update
Moritz Angermann's avatar
Moritz Angermann committed
318 319
----------------

320
``cabal v2-update`` updates the state of the package index. If the
Moritz Angermann's avatar
Moritz Angermann committed
321 322 323
project contains multiple remote package repositories it will update
the index of all of them (e.g. when using overlays).

324
Some examples:
Moritz Angermann's avatar
Moritz Angermann committed
325 326 327

::

328 329
    $ cabal v2-update                  # update all remote repos
    $ cabal v2-update head.hackage     # update only head.hackage
Moritz Angermann's avatar
Moritz Angermann committed
330

331
cabal v2-build
332 333
---------------

334
``cabal v2-build`` takes a set of targets and builds them. It
335 336 337 338 339 340 341 342 343 344 345 346 347 348
automatically handles building and installing any dependencies of these
targets.

A target can take any of the following forms:

-  A package target: ``package``, which specifies that all enabled
   components of a package to be built. By default, test suites and
   benchmarks are *not* enabled, unless they are explicitly requested
   (e.g., via ``--enable-tests``.)

-  A component target: ``[package:][ctype:]component``, which specifies
   a specific component (e.g., a library, executable, test suite or
   benchmark) to be built.

Duncan Coutts's avatar
Duncan Coutts committed
349 350
-  All packages: ``all``, which specifies all packages within the project.

351 352 353 354 355 356 357 358
-  Components of a particular type: ``package:ctypes``, ``all:ctypes``:
   which specifies all components of the given type. Where valid
   ``ctypes`` are:
     - ``libs``, ``libraries``,
     - ``flibs``, ``foreign-libraries``,
     - ``exes``, ``executables``,
     - ``tests``,
     - ``benches``, ``benchmarks``.
Duncan Coutts's avatar
Duncan Coutts committed
359

360
In component targets, ``package:`` and ``ctype:`` (valid component types
361
are ``lib``, ``flib``, ``exe``, ``test`` and ``bench``) can be used to
362 363 364 365 366 367 368 369 370
disambiguate when multiple packages define the same component, or the
same component name is used in a package (e.g., a package ``foo``
defines both an executable and library named ``foo``). We always prefer
interpreting a target as a package name rather than as a component name.

Some example targets:

::

371 372
    $ cabal v2-build lib:foo-pkg       # build the library named foo-pkg
    $ cabal v2-build foo-pkg:foo-tests # build foo-tests in foo-pkg
373 374 375 376

(There is also syntax for specifying module and file targets, but it
doesn't currently do anything.)

377 378
Beyond a list of targets, ``cabal v2-build`` accepts all the flags that
``cabal v2-configure`` takes. Most of these flags are only taken into
379 380 381 382 383
consideration when building local packages; however, some flags may
cause extra store packages to be built (for example,
``--enable-profiling`` will automatically make sure profiling libraries
for all transitive dependencies are built and installed.)

384
In addition ``cabal v2-build`` accepts these flags:
385 386 387 388 389

- ``--only-configure``: When given we will forgoe performing a full build and
  abort after running the configure phase of each target package.


390
cabal v2-repl
391 392
--------------

393 394
``cabal v2-repl TARGET`` loads all of the modules of the target into
GHCi as interpreted bytecode. In addition to ``cabal v2-build``'s flags,
Merijn Verstraaten's avatar
Merijn Verstraaten committed
395
it takes an additional ``--repl-options`` flag.
396 397

To avoid ``ghci`` specific flags from triggering unneeded global rebuilds these
Merijn Verstraaten's avatar
Merijn Verstraaten committed
398 399 400 401 402
flags are now stripped from the internal configuration. As a result
``--ghc-options`` will no longer (reliably) work to pass flags to ``ghci`` (or
other repls). Instead, you should use the new ``--repl-options`` flag to
specify these options to the invoked repl. (This flag also works on ``cabal
repl`` and ``Setup repl`` on sufficiently new versions of Cabal.)
403

404 405
Currently, it is not supported to pass multiple targets to ``v2-repl``
(``v2-repl`` will just successively open a separate GHCi session for
406 407
each target.)

Alexis Williams's avatar
Alexis Williams committed
408 409 410 411 412 413
It also provides a way to experiment with libraries without needing to download
them manually or to install them globally.

This command opens a REPL with the current default target loaded, and a version
of the ``vector`` package matching that specification exposed.

414
::
Alexis Williams's avatar
Alexis Williams committed
415

416
    $ cabal v2-repl --build-depends "vector >= 0.12 && < 0.13"
Alexis Williams's avatar
Alexis Williams committed
417 418

Both of these commands do the same thing as the above, but only exposes ``base``,
419
``vector``, and the ``vector`` package's transitive dependencies even if the user
Alexis Williams's avatar
Alexis Williams committed
420 421 422 423
is in a project context.

::

424 425
    $ cabal v2-repl --ignore-project --build-depends "vector >= 0.12 && < 0.13"
    $ cabal v2-repl --project='' --build-depends "vector >= 0.12 && < 0.13"
Alexis Williams's avatar
Alexis Williams committed
426 427 428 429 430 431 432

This command would add ``vector``, but not (for example) ``primitive``, because
it only includes the packages specified on the command line (and ``base``, which
cannot be excluded for technical reasons).

::

433
    $ cabal v2-repl --build-depends vector --no-transitive-deps
Alexis Williams's avatar
Alexis Williams committed
434

435
cabal v2-run
Francesco Gazzetta's avatar
Francesco Gazzetta committed
436 437
-------------

438
``cabal v2-run [TARGET [ARGS]]`` runs the executable specified by the
Francesco Gazzetta's avatar
Francesco Gazzetta committed
439 440
target, which can be a component, a package or can be left blank, as
long as it can uniquely identify an executable within the project.
441
Tests and benchmarks are also treated as executables.
Francesco Gazzetta's avatar
Francesco Gazzetta committed
442

443
See `the v2-build section <#cabal-new-build>`__ for the target syntax.
Francesco Gazzetta's avatar
Francesco Gazzetta committed
444 445 446 447 448 449 450 451 452 453

Except in the case of the empty target, the strings after it will be
passed to the executable as arguments.

If one of the arguments starts with ``-`` it will be interpreted as
a cabal flag, so if you need to pass flags to the executable you
have to separate them with ``--``.

::

454
    $ cabal v2-run target -- -a -bcd --argument
Francesco Gazzetta's avatar
Francesco Gazzetta committed
455

456
'v2-run' also supports running script files that use a certain format. With
Alexis Williams's avatar
Alexis Williams committed
457 458 459 460 461 462 463 464 465 466 467 468 469 470 471 472 473 474 475
a script that looks like:

::

    #!/usr/bin/env cabal
    {- cabal:
    build-depends: base ^>= 4.11
                , shelly ^>= 1.8.1
    -}

    main :: IO ()
    main = do
        ...

It can either be executed like any other script, using ``cabal`` as an
interpreter, or through this command:

::

476 477
    $ cabal v2-run script.hs
    $ cabal v2-run script.hs -- --arg1 # args are passed like this
Alexis Williams's avatar
Alexis Williams committed
478

479
cabal v2-freeze
480 481
----------------

482
``cabal v2-freeze`` writes out a **freeze file** which records all of
483 484 485 486 487 488 489 490
the versions and flags which that are picked by the solver under the
current index and flags.  Default name of this file is
``cabal.project.freeze`` but in combination with a
``--project-file=my.project`` flag (see :ref:`project-file
<cmdoption-project-file>`)
the name will be ``my.project.freeze``.
A freeze file has the same syntax as ``cabal.project`` and looks
something like this:
491

Leonid Onokhov's avatar
Leonid Onokhov committed
492
.. highlight:: cabal
493

Leonid Onokhov's avatar
Leonid Onokhov committed
494
::
Leonid Onokhov's avatar
Leonid Onokhov committed
495

496 497 498 499
    constraints: HTTP ==4000.3.3,
                 HTTP +warp-tests -warn-as-error -network23 +network-uri -mtl1 -conduit10,
                 QuickCheck ==2.9.1,
                 QuickCheck +templatehaskell,
500 501
                 -- etc...

502 503 504 505 506 507 508

For end-user executables, it is recommended that you distribute the
``cabal.project.freeze`` file in your source repository so that all
users see a consistent set of dependencies. For libraries, this is not
recommended: users often need to build against different versions of
libraries than what you developed against.

509
cabal v2-bench
510 511
---------------

512
``cabal v2-bench [TARGETS] [OPTIONS]`` runs the specified benchmarks
513 514 515
(all the benchmarks in the current package by default), first ensuring
they are up to date.

516
cabal v2-test
517 518
--------------

519
``cabal v2-test [TARGETS] [OPTIONS]`` runs the specified test suites
520 521 522
(all the test suites in the current package by default), first ensuring
they are up to date.

523
cabal v2-haddock
524 525
-----------------

526
``cabal v2-haddock [FLAGS] [TARGET]`` builds Haddock documentation for
527 528
the specified packages within the project.

alexbiehl's avatar
alexbiehl committed
529 530 531 532
If a target is not a library :cfg-field:`haddock-benchmarks`,
:cfg-field:`haddock-executables`, :cfg-field:`haddock-internal`,
:cfg-field:`haddock-tests` will be implied as necessary.

533
cabal v2-exec
Francesco Gazzetta's avatar
Francesco Gazzetta committed
534
---------------
535

536
``cabal v2-exec [FLAGS] [--] COMMAND [--] [ARGS]`` runs the specified command
Francesco Gazzetta's avatar
Francesco Gazzetta committed
537 538
using the project's environment. That is, passing the right flags to compiler
invocations and bringing the project's executables into scope.
539

540
cabal v2-install
Francesco Gazzetta's avatar
Francesco Gazzetta committed
541 542
-----------------

543
``cabal v2-install [FLAGS] PACKAGES`` builds the specified packages and
544
symlinks/copies their executables in ``installdir`` (usually ``~/.cabal/bin``).
Francesco Gazzetta's avatar
Francesco Gazzetta committed
545 546 547 548 549 550

For example this command will build the latest ``cabal-install`` and symlink
its ``cabal`` executable:

::

551
    $ cabal v2-install cabal-install
Francesco Gazzetta's avatar
Francesco Gazzetta committed
552

553
In addition, it's possible to use ``cabal v2-install`` to install components
554 555 556 557 558 559
of a local project. For example, with an up-to-date Git clone of the Cabal
repository, this command will build cabal-install HEAD and symlink the
``cabal`` executable:

::

560
    $ cabal v2-install exe:cabal
561

562 563 564 565 566
Where symlinking is not possible (eg. on Windows), ``--install-method=copy``
can be used:

::

567
    $ cabal v2-install exe:cabal --install-method=copy --installdir=~/bin
568 569 570 571

Note that copied executables are not self-contained, since they might use
data-files from the store.

572
It is also possible to "install" libraries using the ``--lib`` flag. For
573 574 575 576
example, this command will build the latest Cabal library and install it:

::

577
    $ cabal v2-install --lib Cabal
578 579 580

This works by managing GHC environments. By default, it is writing to the
global environment in ``~/.ghc/$ARCH-$OS-$GHCVER/environments/default``.
581
``v2-install`` provides the ``--package-env`` flag to control which of
582 583 584 585 586 587
these environments is modified.

This command will modify the environment file in the current directory:

::

588
    $ cabal v2-install --lib Cabal --package-env .
589

Leon Schoorl's avatar
Leon Schoorl committed
590
This command will modify the environment file in the ``~/foo`` directory:
591 592 593

::

594
    $ cabal v2-install --lib Cabal --package-env foo/
595 596

Do note that the results of the previous two commands will be overwritten by
597
the use of other v2-style commands, so it is not recommended to use them inside
598 599 600 601 602 603 604
a project directory.

This command will modify the environment in the "local.env" file in the
current directory:

::

605
    $ cabal v2-install --lib Cabal --package-env local.env
606 607 608 609 610

This command will modify the ``myenv`` named global environment:

::

611
    $ cabal v2-install --lib Cabal --package-env myenv
612 613 614 615 616 617

If you wish to create a named environment file in the current directory where
the name does not contain an extension, you must reference it as ``./myenv``.

You can learn more about how to use these environments in `this section of the
GHC manual <https://downloads.haskell.org/~ghc/latest/docs/html/users_guide/packages.html#package-environments>`_.
Francesco Gazzetta's avatar
Francesco Gazzetta committed
618

619
cabal v2-clean
620 621
---------------

622
``cabal v2-clean [FLAGS]`` cleans up the temporary files and build artifacts
623 624 625 626 627 628 629
stored in the ``dist-newstyle`` folder.

By default, it removes the entire folder, but it can also spare the configuration
and caches if the ``--save-config`` option is given, in which case it only removes
the build artefacts (``.hi``, ``.o`` along with any other temporary files generated
by the compiler, along with the build output).

630
cabal v2-sdist
631 632
---------------

633
``cabal v2-sdist [FLAGS] [TARGETS]`` takes the crucial files needed to build ``TARGETS``
634 635 636
and puts them into an archive format ready for upload to Hackage. These archives are stable
and two archives of the same format built from the same source will hash to the same value.

637
``cabal v2-sdist`` takes the following flags:
638

639
- ``-l``, ``--list-only``: Rather than creating an archive, lists files that would be included.
640 641
  Output is to ``stdout`` by default. The file paths are relative to the project's root
  directory.
642

643 644 645 646 647 648
- ``-o``, ``--output-dir``: Sets the output dir, if a non-default one is desired. The default is
  ``dist-newstyle/sdist/``. ``--output-dir -`` will send output to ``stdout``
  unless multiple archives are being created.

- ``-z``, ``--null``: Only used with ``--list-only``. Separates filenames with a NUL
  byte instead of newlines.
649

650
``v2-sdist`` is inherently incompatible with sdist hooks, not due to implementation but due
Alexis Williams's avatar
Alexis Williams committed
651
to fundamental core invariants (same source code should result in the same tarball, byte for
652
byte) that must be satisfied for it to function correctly in the larger v2-build ecosystem.
Alexis Williams's avatar
Alexis Williams committed
653 654 655
``autogen-modules`` is able to replace uses of the hooks to add generated modules, along with
the custom publishing of Haddock documentation to Hackage.

656 657 658 659 660 661 662 663
.. warning::

  Packages that use Backpack will stop working if uploaded to
  Hackage, due to `issue #6005 <https://github.com/haskell/cabal/issues/6005>`_.
  While this is happening, we recommend not uploading these packages
  to Hackage (and instead referencing the package directly
  as a ``source-repository-package``).

664 665 666 667 668 669 670 671 672 673 674 675 676 677 678 679 680 681
Configuring builds with cabal.project
=====================================

``cabal.project`` files support a variety of options which configure the
details of your build. The general syntax of a ``cabal.project`` file is
similar to that of a Cabal file: there are a number of fields, some of
which live inside stanzas:

::

    packages: */*.cabal
    with-compiler: /opt/ghc/8.0.1/bin/ghc

    package cryptohash
      optimization: False

In general, the accepted field names coincide with the accepted command
line flags that ``cabal install`` and other commands take. For example,
682
``cabal v2-configure --enable-profiling`` will write out a project
683
file with ``profiling: True``.
684 685 686 687 688 689

The full configuration of a project is determined by combining the
following sources (later entries override earlier ones):

1. ``~/.cabal/config`` (the user-wide global configuration)

Leon Schoorl's avatar
Leon Schoorl committed
690
2. ``cabal.project`` (the project configuration)
691

692
3. ``cabal.project.freeze`` (the output of ``cabal v2-freeze``)
693

694
4. ``cabal.project.local`` (the output of ``cabal v2-configure``)
695

696

697 698 699 700 701 702
Specifying the local packages
-----------------------------

The following top-level options specify what the local packages of a
project are:

703
.. cfg-field:: packages: package location list (space or comma separated)
Leonid Onokhov's avatar
Leonid Onokhov committed
704
    :synopsis: Project packages.
705 706

    :default: ``./*.cabal``
707

708 709 710 711 712
    Specifies the list of package locations which contain the local
    packages to be built by this project. Package locations can take the
    following forms:

    1. They can specify a Cabal file, or a directory containing a Cabal
713
       file, e.g., ``packages: Cabal cabal-install/cabal-install.cabal``.
714 715 716

    2. They can specify a glob-style wildcards, which must match one or
       more (a) directories containing a (single) Cabal file, (b) Cabal
717 718 719
       files (extension ``.cabal``), or (c) tarballs which contain Cabal
       packages (extension ``.tar.gz``).
       For example, to match all Cabal files in all
720 721 722 723
       subdirectories, as well as the Cabal projects in the parent
       directories ``foo`` and ``bar``, use
       ``packages: */*.cabal ../{foo,bar}/``

724
    3. They can specify an ``http``, ``https`` or ``file``
725
       URL, representing the path to a remote tarball to be downloaded
726
       and built.
727

728
    There is no command line variant of this field; see :issue:`3585`.
729

730
.. cfg-field:: optional-packages: package location list (space or comma-separated)
Leonid Onokhov's avatar
Leonid Onokhov committed
731
    :synopsis: Optional project packages.
732 733 734

    :default: ``./*/*.cabal``

Leonid Onokhov's avatar
Leonid Onokhov committed
735 736 737 738 739 740
    Like :cfg-field:`packages`, specifies a list of package locations
    containing local packages to be built. Unlike :cfg-field:`packages`,
    if we glob for a package, it is permissible for the glob to match against
    zero packages. The intended use-case for :cfg-field:`optional-packages`
    is to make it so that vendored packages can be automatically picked up if
    they are placed in a subdirectory, but not error if there aren't any.
741 742 743

    There is no command line variant of this field.

744
.. cfg-field:: extra-packages: package list with version bounds (comma separated)
Leonid Onokhov's avatar
Leonid Onokhov committed
745
    :synopsis: Adds external pacakges as local
746

747 748 749 750 751
    [STRIKEOUT:Specifies a list of external packages from Hackage which
    should be considered local packages.] (Not implemented)

    There is no command line variant of this field.

752

753

754 755 756 757 758 759 760 761 762 763 764 765 766 767 768 769
All local packages are *vendored*, in the sense that if other packages
(including external ones from Hackage) depend on a package with the name
of a local package, the local package is preferentially used.  This
motivates the default settings::

    packages: ./*.cabal
    optional-packages: ./*/*.cabal

...any package can be vendored simply by making a checkout in the
top-level project directory, as might be seen in this hypothetical
directory layout::

    foo.cabal
    foo-helper/     # local package
    unix/           # vendored external package

770
All of these options support globs. ``cabal v2-build`` has its own glob
771 772 773 774 775 776 777 778 779 780 781 782 783 784 785
format:

-  Anywhere in a path, as many times as you like, you can specify an
   asterisk ``*`` wildcard. E.g., ``*/*.cabal`` matches all ``.cabal``
   files in all immediate subdirectories. Like in glob(7), asterisks do
   not match hidden files unless there is an explicit period, e.g.,
   ``.*/foo.cabal`` will match ``.private/foo.cabal`` (but
   ``*/foo.cabal`` will not).

-  You can use braces to specify specific directories; e.g.,
   ``{vendor,pkgs}/*.cabal`` matches all Cabal files in the ``vendor``
   and ``pkgs`` subdirectories.

Formally, the format described by the following BNF:

786 787 788
.. todo::
    convert globbing grammar to proper ABNF_ syntax

789
.. code-block:: abnf
790 791 792 793 794 795 796 797 798 799 800 801 802 803 804 805

    FilePathGlob    ::= FilePathRoot FilePathGlobRel
    FilePathRoot    ::= {- empty -}        # relative to cabal.project
                      | "/"                # Unix root
                      | [a-zA-Z] ":" [/\\] # Windows root
                      | "~"                # home directory
    FilePathGlobRel ::= Glob "/"  FilePathGlobRel # Unix directory
                      | Glob "\\" FilePathGlobRel # Windows directory
                      | Glob         # file
                      | {- empty -}  # trailing slash
    Glob      ::= GlobPiece *
    GlobPiece ::= "*"            # wildcard
                | [^*{},/\\] *   # literal string
                | "\\" [*{},]    # escaped reserved character
                | "{" Glob "," ... "," Glob "}" # union (match any of these)

806 807 808 809 810 811 812 813 814 815 816 817 818 819 820 821 822 823 824 825 826 827 828 829 830 831 832 833 834 835

Specifying Packages from Remote Version Control Locations
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Starting with Cabal 2.4, there is now a stanza
``source-repository-package`` for specifying packages from an external
version control which supports the following fields:

- :pkg-field:`source-repository:type`
- :pkg-field:`source-repository:location`
- :pkg-field:`source-repository:tag`
- :pkg-field:`source-repository:subdir`

A simple example is shown below:

.. code-block:: cabal

    packages: .

    source-repository-package
        type: git
        location: https://github.com/hvr/HsYAML.git
        tag: e70cf0c171c9a586b62b3f75d72f1591e4e6aaa1

    source-repository-package
        type: git
        location: https://github.com/well-typed/cborg
        tag: 3d274c14ca3077c3a081ba7ad57c5182da65c8c1
        subdir: cborg

836 837 838 839 840 841
Global configuration options
----------------------------

The following top-level configuration options are not specific to any
package, and thus apply globally:

842

843 844
.. cfg-field:: verbose: nat
               --verbose=n, -vn
Leonid Onokhov's avatar
Leonid Onokhov committed
845
    :synopsis: Build verbosity level.
846 847 848

    :default: 1

849 850 851 852 853 854
    Control the verbosity of ``cabal`` commands, valid values are from 0
    to 3.

    The command line variant of this field is ``--verbose=2``; a short
    form ``-v2`` is also supported.

855 856
.. cfg-field:: jobs: nat or $ncpus
               --jobs=n, -jn, --jobs=$ncpus
Leonid Onokhov's avatar
Leonid Onokhov committed
857
    :synopsis: Number of builds running in parallel.
858 859 860

    :default: 1

861 862 863 864 865 866 867 868 869
    Run *nat* jobs simultaneously when building. If ``$ncpus`` is
    specified, run the number of jobs equal to the number of CPUs.
    Package building is often quite parallel, so turning on parallelism
    can speed up build times quite a bit!

    The command line variant of this field is ``--jobs=2``; a short form
    ``-j2`` is also supported; a bare ``--jobs`` or ``-j`` is equivalent
    to ``--jobs=$ncpus``.

870 871
.. cfg-field::  keep-going: boolean
                --keep-going
Leonid Onokhov's avatar
Leonid Onokhov committed
872
    :synopsis: Try to continue building on failure.
873 874 875

    :default: False

876 877 878 879 880
    If true, after a build failure, continue to build other unaffected
    packages.

    The command line variant of this field is ``--keep-going``.

881 882 883 884 885 886 887 888 889 890
.. option:: --builddir=DIR

    Specifies the name of the directory where build products for
    build will be stored; defaults to ``dist-newstyle``.  If a
    relative name is specified, this directory is resolved relative
    to the root of the project (i.e., where the ``cabal.project``
    file lives.)

    This option cannot be specified via a ``cabal.project`` file.

891
.. _cmdoption-project-file:
892 893 894 895 896 897 898 899 900 901 902 903 904 905 906 907
.. option:: --project-file=FILE

    Specifies the name of the project file used to specify the
    rest of the top-level configuration; defaults to ``cabal.project``.
    This name not only specifies the name of the main project file,
    but also the auxiliary project files ``cabal.project.freeze``
    and ``cabal.project.local``; for example, if you specify
    ``--project-file=my.project``, then the other files that will
    be probed are ``my.project.freeze`` and ``my.project.local``.

    If the specified project file is a relative path, we will
    look for the file relative to the current working directory,
    and then for the parent directory, until the project file is
    found or we have hit the top of the user's home directory.

    This option cannot be specified via a ``cabal.project`` file.
908

Alex Biehl's avatar
Alex Biehl committed
909 910 911
.. option:: --store-dir=DIR

    Specifies the name of the directory of the global package store.
912

913 914 915 916 917
Solver configuration options
----------------------------

The following settings control the behavior of the dependency solver:

918
.. cfg-field:: constraints: constraints list (comma separated)
919
               --constraint="pkg >= 2.0"
Leonid Onokhov's avatar
Leonid Onokhov committed
920
    :synopsis: Extra dependencies constraints.
921

922 923 924
    Add extra constraints to the version bounds, flag settings,
    and other properties a solver can pick for a
    package. For example:
925

926 927 928 929 930 931 932 933 934 935
    ::

        constraints: bar == 2.1

    A package can be specified multiple times in ``constraints``, in
    which case the specified constraints are intersected. This is
    useful, since the syntax does not allow you to specify multiple
    constraints at once. For example, to specify both version bounds and
    flag assignments, you would write:

936 937 938
    ::

        constraints: bar == 2.1,
939
                     bar +foo -baz
940

941 942
    Valid constraints take the same form as for the `constraint
    command line option
943
    <installing-packages.html#cmdoption-setup-configure-constraint>`__.
944

945 946
.. cfg-field:: preferences: preference (comma separated)
               --preference="pkg >= 2.0"
Leonid Onokhov's avatar
Leonid Onokhov committed
947
    :synopsis: Prefered dependency versions.
948 949 950

    Like :cfg-field:`constraints`, but the solver will attempt to satisfy
    these preferences on a best-effort basis. The resulting install is locally
951 952 953 954 955 956 957 958
    optimal with respect to preferences; specifically, no single package
    could be replaced with a more preferred version that still satisfies
    the hard constraints.

    Operationally, preferences can cause the solver to attempt certain
    version choices of a package before others, which can improve
    dependency solver runtime.

959
    One way to use :cfg-field:`preferences` is to take a known working set of
960
    constraints (e.g., via ``cabal v2-freeze``) and record them as
961 962 963 964 965 966 967 968 969
    preferences. In this case, the solver will first attempt to use this
    configuration, and if this violates hard constraints, it will try to
    find the minimal number of upgrades to satisfy the hard constraints
    again.

    The command line variant of this field is
    ``--preference="pkg >= 2.0"``; to specify multiple preferences, pass
    the flag multiple times.

970
.. cfg-field:: allow-newer: none, all or list of scoped package names (space or comma separated)
971
               --allow-newer, --allow-newer=[none,all,[scope:][^]pkg]
Brian Wignall's avatar
Brian Wignall committed
972
    :synopsis: Lift dependencies upper bound constraints.
973 974 975

    :default: ``none``

976
    Allow the solver to pick an newer version of some packages than
977 978
    would normally be permitted by than the :pkg-field:`build-depends` bounds
    of packages in the install plan. This option may be useful if the
979 980
    dependency solver cannot otherwise find a valid install plan.

981
    For example, to relax ``pkg``\ s :pkg-field:`build-depends` upper bound on
982 983 984 985 986 987
    ``dep-pkg``, write a scoped package name of the form:

    ::

        allow-newer: pkg:dep-pkg

988 989 990 991 992 993 994 995 996 997 998
    If the scope shall be limited to specific releases of ``pkg``, the
    extended form as in

    ::

        allow-newer: pkg-1.2.3:dep-pkg, pkg-1.1.2:dep-pkg

    can be used to limit the relaxation of dependencies on
    ``dep-pkg`` by the ``pkg-1.2.3`` and ``pkg-1.1.2`` releases only.

    The scoped syntax is recommended, as it is often only a single package
999 1000 1001 1002
    whose upper bound is misbehaving. In this case, the upper bounds of
    other packages should still be respected; indeed, relaxing the bound
    can break some packages which test the selected version of packages.

1003 1004 1005 1006 1007 1008 1009
    The syntax also allows to prefix the dependee package with a
    modifier symbol to modify the scope/semantic of the relaxation
    transformation in a additional ways. Currently only one modifier
    symbol is defined, i.e. ``^`` (i.e. caret) which causes the
    relaxation to be applied only to ``^>=`` operators and leave all other
    version operators untouched.

1010 1011 1012 1013 1014 1015 1016 1017
    However, in some situations (e.g., when attempting to build packages
    on a new version of GHC), it is useful to disregard *all*
    upper-bounds, with respect to a package or all packages. This can be
    done by specifying just a package name, or using the keyword ``all``
    to specify all packages:

    ::

Leonid Onokhov's avatar
Leonid Onokhov committed
1018
        -- Disregard upper bounds involving the dependencies on
1019 1020 1021
        -- packages bar, baz. For quux only, relax
        -- 'quux ^>= ...'-style constraints only.
        allow-newer: bar, baz, ^quux
1022

Leonid Onokhov's avatar
Leonid Onokhov committed
1023
        -- Disregard all upper bounds when dependency solving
1024 1025
        allow-newer: all

1026 1027 1028
        -- Disregard all `^>=`-style upper bounds when dependency solving
        allow-newer: ^all

1029 1030

    For consistency, there is also the explicit wildcard scope syntax
1031 1032
    ``*`` (or its alphabetic synonym ``all``). Consequently, the
    examples above are equivalent to the explicitly scoped variants:
1033 1034 1035 1036 1037

    ::

        allow-newer: all:bar, *:baz, *:^quux

1038 1039 1040 1041 1042 1043 1044 1045 1046 1047 1048 1049 1050 1051 1052 1053 1054 1055
        allow-newer: *:*
        allow-newer: all:all

        allow-newer: *:^*
        allow-newer: all:^all

    In order to ignore all bounds specified by a package ``pkg-1.2.3``
    you can combine scoping with a right-hand-side wildcard like so

    ::

        -- Disregard any upper bounds specified by pkg-1.2.3
        allow-newer: pkg-1.2.3:*

        -- Disregard only `^>=`-style upper bounds in pkg-1.2.3
        allow-newer: pkg-1.2.3:^*


1056 1057 1058
    :cfg-field:`allow-newer` is often used in conjunction with a constraint
    (in the cfg-field:`constraints` field) forcing the usage of a specific,
    newer version of a package.
1059

1060
    The command line variant of this field is e.g. ``--allow-newer=bar``. A
1061 1062
    bare ``--allow-newer`` is equivalent to ``--allow-newer=all``.

1063
.. cfg-field:: allow-older: none, all, list of scoped package names (space or comma separated)
1064
               --allow-older, --allow-older=[none,all,[scope:][^]pkg]
Brian Wignall's avatar
Brian Wignall committed
1065
    :synopsis: Lift dependency lower bound constraints.
1066
    :since: 2.0
1067 1068 1069 1070 1071

    :default: ``none``

    Like :cfg-field:`allow-newer`, but applied to lower bounds rather than
    upper bounds.
1072 1073 1074 1075

    The command line variant of this field is ``--allow-older=all``. A
    bare ``--allow-older`` is equivalent to ``--allow-older=all``.

1076 1077 1078

.. cfg-field:: index-state: HEAD, unix-timestamp, ISO8601 UTC timestamp.
   :synopsis: Use source package index state as it existed at a previous time.
1079
   :since: 2.0
1080 1081 1082 1083 1084 1085 1086 1087 1088 1089 1090 1091 1092 1093

   :default: ``HEAD``

   This allows to change the source package index state the solver uses
   to compute install-plans. This is particularly useful in
   combination with freeze-files in order to also freeze the state the
   package index was in at the time the install-plan was frozen.

   ::

      -- UNIX timestamp format example
      index-state: @1474739268

      -- ISO8601 UTC timestamp format example
1094
      -- This format is used by 'cabal v2-configure'
1095 1096 1097 1098
      -- for storing `--index-state` values.
      index-state: 2016-09-24T17:47:48Z


1099 1100 1101 1102 1103 1104 1105 1106 1107 1108 1109 1110 1111 1112 1113 1114
.. cfg-field:: reject-unconstrained-dependencies: all, none
               --reject-unconstrained-dependencies=[all|none]
   :synopsis: Restrict the solver to packages that have constraints on them.

   :default: none
   :since: 2.6

   By default, the dependency solver can include any package that it's
   aware of in a build plan. If you wish to restrict the build plan to
   a closed set of packages (e.g., from a freeze file), use this flag.

   When set to `all`, all non-local packages that aren't goals must be
   explicitly constrained. When set to `none`, the solver will
   consider all packages.


1115 1116 1117
Package configuration options
-----------------------------

1118
Package options affect the building of specific packages. There are three
1119 1120 1121 1122 1123 1124 1125 1126 1127
ways a package option can be specified:

-  They can be specified at the top-level, in which case they apply only
   to **local package**, or

-  They can be specified inside a ``package`` stanza, in which case they
   apply to the build of the package, whether or not it is local or
   external.

1128
-  They can be specified inside an ``package *`` stanza, in which case they
1129 1130 1131 1132
   apply to all packages, local ones from the project and also external
   dependencies.