- Use real-life example from Cabal project in doctest

This allows us to compile Setup.hs without depending on base.
In particular, this ensures that a package with `build-type: Hooks`
and a custom setup stanza that does not depend on base successfully
compiles.

Tested in PackageTests/SetupHooks/SetupHooksNoBase.

This commit initialises the Haddock flag haddockKeepTempFiles with the
value of buildSettingKeepTempFiles.

This addresses an 8 year old TODO in the code, from Duncan Coutts.

This makes Setup copy/install succeed if there's
nothing to do because the package doesn't contain
a library or executable.

This allows downstream users of Cabal to avoid having to
add workarounds for this edge case.

Resolves #6750

Previously `--package-db` was only passed to test which used the `v2-`
prefix.

Now we pass `--package-db` to things which use the `v2-` prefix, not
things which use the `v1-` prefix and by default assume that unprefixed
commands are v2 commands.

The --working-dir flag is only available for Cabal >= 3.13.
This commit fixes an incorrect conditional: we only filtered out this
flag for Cabal < 3.11, instead of < 3.13.

Test: PackageTests/WorkingDir

Fixes #9940

This patch ensures that we correctly provision executables declared
in the build-tool-depends fields in all circumstances:

  - whether the build tool is external (from another package) or
    internal (declared in the current package)
  - whether the build tool is used at compile time (e.g. in a pre-build
    rule or in a Template Haskell splice) or at run time (e.g. when
    running a test-suite, benchmark or executable).

Note that correctly provisioning a build tool requires two pieces of
information:

  - making it available in PATH,
  - ensuring it has the correct environment variables overrides;
    in particular, the build tool needs to be able to find its own
    data directory.

The test case BuildToolPaths checks all of these situations are handled
correctly.

Matthew Pickering authored 1 year ago and Rodrigo Mesquita committed 11 months ago

This makes `withShorterPathForNewBuildStore` fit more nicely into the
rest of the testing infrastructure.

* Move `withShorterPathForNewBuildStore` to `TestM` monad
* Move responsibility for passing `--store-dir` to `cabalGArgs` function
* Move `findDependencyInStore` into `TestM`, and remove requirement to
  pass path to store directory.
* Introduce `testStoreDir` function which returns the store location
  (and honours `withShorterPathForNewBuildStore`)
* Migrate tests which use `withShorterPathForNewBuildStore`.

This commit introduces a new build-type, Hooks. A package using this
build type should provide a SetupHooks.hs module which exports
a value `setupHooks :: SetupHooks`. This is intended to replace the
Custom setup type. This allows Cabal to have finer-grained information
about the build, instead of having an opaque Setup executable to invoke.

Tests include:

  - error when returning an invalid component diff in a
    per-component pre-configure hook
  - error when declaring pre-build rules whose dependency
    graph contains cycles
  - error when we cannot find a dependency of a pre-build rule
  - warning when there are pre-build rules that are declared but
    never demanded
  - correctness tests for SetupHooks, e.g. that
    pre-build rules are run in dependency order (see the
    `SetupHooksRuleOrdering` test)

Add tests for #9799 about freeze qualifying all packages with 'any'
constraint scope.

This reverts commit a90d44ff.

* Redesign 'cabal path' command to account for projects

Previously, `cabal path` was only able to query from the global
configuration file, e.g., `~/.cabal/config` or the XDG equivalent.
Adding support for cabal project is a huge boost to usability.

We take the foundations and turn them into `cabal v2-path` which takes
project configuration, such as `cabal.project` into account.
Note, the command is still named `cabal path`, but for the sake of
disambiguation, we refer to this new iteration of the command as `cabal
v2-path`.

In addition, we add support for multiple output formats, such as
key-value pairs and json.

The key-value pair output prints a line for each queried key and its
respective value:

    key1: value2
    key2: value2

If only a single key is queried, we print only the
value, for example:

    value1

The json output format is versioned by the cabal-install version which
is part of the json object.
Thus, all result objects contain at least the key "cabal-install-version".

We expand the `cabal v2-path` to also produce information of the
compiler that is going to be used in a `cabal build` or `cabal repl` invocation.
To do that, we rebuild the install plan and query for the configured
compiler program.
This is helpful for downstream tools, such as HLS, to figure out the GHC
version required to compile a project with.

We also add an exhaustive test suite for 'cabal path' cmd

We test that each query honours cabal.project files, cli parameters, and
is composable with the other query flags.

We extend the test output normalisers for ghc compiler location and
cabal-install version, as the 'cabal path' command outputs the exact ghc
and ghc-pkg location. In addition, the json output format is versioned
on the cabal-install version.

Currently, we query the cabal-install version on each test normalisation
run. This might be unnecessary expensive, and could be avoided by
introducing a 'cabalProgram' that specifies how the program version can
be found. This way, we can cache the version query.

Add '--cache-home' flag thats shows the cabal's cache root

Rename '--cache-dir' to the more correct '--remote-repo-dir'.

* Update 'cabal path' documentation

* Add changelog.d entry

---------

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

With this change to the solver message rendering, I also fix some bugs around project imports, adding tests for those cases. Reviewers asked that the Y-shaped import checks (using IORef) be made on a separate pull request. Removing those lead to cascading deletions.

- Regenerate expected .out files
- Show tree provenance of import constraint
- Add trimmed down PackageTests/VersionPriority
- Add changelog entry
- Use NonEmpty
- Fix check for cyclical import
- Use primes for next iteration
- Remove unused LANGUAGE pragmas
- Rename to projectConfigPathRoot
- Docs for ProjectConfigPath and showProjectConfigPath
- Renaming
- Add cyclical import tests with 1 and 2 hops in cycle
- Use full path for cyclical error message
- Expected output has project with full project path
- Add fullPath local function
- Project directory as FilePath, not Maybe FilePath
- Use (_, projectFileName) binding splitFileName
- Need full path to project parsing legacy
- Inline seenImports conversion
- Add cyclical checks with same file names and hops
- Add noncyclical tests that hop over folders
- Add a project testing skipping in and out of a folder
- Update expectations of cyclical tests
- Use canonicalizePath for collapsing .. when possible
- Capture trace for later
- Add module for ProjectConfigPath
- Move functions for ProjectConfigPath to its module
- Fetch URI is not prefixed with ./https://etc
- Document normaliseConfigPath
- Add doctests for normaliseConfigPath
- Add doctest of canonicalizeConfigPath
- Show an example of canonical paths
- Use importer and importee in canonicalizeConfigPaths
- Add logging
- Use normLocPath, drop -XMultiWayIf
- Remove seenImports parameter from parseProject
- Follow hlint suggestion: Move brackets to avoid $
- Follow hlint warning: Use mapM
- Follow hlint warning: Use traverse_
- Follow hlint suggestion: Use <$>
- Add failing test skipping across folder
- Remove normaliseConfigPath
- Add hasDuplicatesConfigPath
- Remove lengthConfigPath and nubConfigPath
- Mention cabal-install-solver in the changelog
- Left align project import tree with other content
- Rerun tests to generate left-aligned tree output
- Don't show "constraint from project requires ..."
- Regenerate test outputs without (constraint from project ...)
- Move fail reason printing to project config path module
- Minimize diff of showFR
- Show an "imported by: " list instead of a tree
- Update info logging, avoiding leading spaces
- When logging I found leading spaces were trimmed
- Don't log relative location and path
- Use unicode tree symbol in haddocks
- A duplicate importing tests
- Add Y-forking import test
- A test for detecting when the same config is imported via many different paths
- Fix a typo with ../noncyclical-same-filename-b.config
- Add an intercepting cabal-testsuite/cabal.project
- Use IORef to detect seen imports
- Get rid of seenImports
- Use IORef [FilePath] of uniqueImports
- Add consProjectConfigPath
- Expand the haddocks of newtype ProjectConfigPath
- Clarify, it's the project root directory
- Rename a function to makeRelativeConfigPath
- Fix whitespace
- Update output snippet in changelog
- Put the info logging together
- Split cycles and duplicates detection
- Add fixes and duplicate import to the change log
- Better reporting of duplicate imports
- Report cycles and duplicates using unique file name
- This is after canonicalizeConfigPath, relative to the project directory
- Get rid of fullLocPath, use normLocPath only
- Do reporting using paths relative to the directory of the project
- Issue a warning for Y-shaped duplicate imports
- Add duplicateImportMsg
- Use pretty printing for duplicate import message
- Use pretty printing for all ProjectConfigPath printing
- Drop a parameter from duplicateImportMsg
- Correct haddocks on docProjectConfigPath
- Satisfy fourmolu
- Remove the surplus space in "imported by "
- Log project parsing at the debug level, up from info
- Simplify the changelog snippet
- Use normalized paths for solver messages
- Satisfy fourmolu
- Relative to the directory of the project
- Duplicate imports are reported as warnings
- No need to mention v2-build
- Remove IORef
- Subdue a check for duplicate imports
- TODO: Catch duplicates across different import paths. We know how to do this
- Remove check for duplicate imports
- Improve the canonicalizeConfigPath doctests
- Add haddocks, drop configPath from go
- Make importsBy NonEmpty
- Use testDir in doctests
- Use canonical test directory
- Regenerate expected outputs
- Assume head of the path is the duplicate
- Get rid of duplicateImportMsg
- Remove unused imports in ProjectConfigPath
- Follow hlint suggestion: use fewer imports
- Remove fullConfigPathRoot
- Follow hlint suggestion: redundant bracket
- Use the original "source" variable name
- Put the source next to its contents
- Go with the original "seenImports" variable name
- Avoid -XMultiWayIf
- Not needed since we aren't checking for duplicates right now
- Be consistent in naming sources
- Move project dir alongside related params
- Rename to pathRequiresVersion
- Add back "constraint from"
- Talk only of cycles and not duplicates
- Don't report line number for cyclical import
- Remove 2nd example in changelog
- Remove Y-forked duplicate import test
- Add "imported by" to assertOutputContains
- Cull excess doctests of canonicalizeConfigPath
- Add deep path doctest
- Update haddocks of canonicalizeConfigPath
- remove idempotent speculation (it isn't idempotent)
- use "indirection" to describe "." and ".."
- makes the path relative to the given directory
- Remove seenImports
- Use Explicit ProjectConfigPath
- Missed one "constraint from" in changelog example
- Don't check (verbosity >= verbose) separately
- While the same level of verbosity as info, it is clearer to not check the level separately.
- Might calculate the path but not use it.
- Add AbsoluteDir
- Keep provenance normalised for display
- Remove stale REVIEW comment
- Storing provenance normalized avoids this
- Fix whitespace
- Satisfy HLint warning: Redundant flip
- Gut the repo/hashable packages
- Revert AbsoluteDir to FilePath

This commit makes the library functions in Cabal agnostic of the working
directory. In practice, this means that we distinguish `FilePath`s
from un-interpreted `SymbolicPath`s. The latter may be paths that are
relative to the working directory, and need to be interpreted with
respect to a passed-in argument specifying the working directory,
instead of using the working directory of the current process.
See Note [Symbolic paths] in Distribution.Utils.Path.

In particular, paths in the package description now use the SymbolicPath
abstraction, which allows specifying whether they are allowed to be
absolute, and, if they are relative, what they are relative to.
For example, source files are relative to a source search directory,
data files are relative to the data directory, and doc files are
relative to the package root.

Fixes #9702

As a warning (“Hackage would reject this package”), to prepare users
to GHC language editions.

- Fix whitespace in cabal-testsuite + other tests

* Add tests for #9742

`main-is` not picked up when inside a multibranch CondNode.

* Fix comments

* Add simplifyBranch to Distribution.Types.CondTree

Goes hand in hand with with simplifyCondTree.

* Make `check` deal correctly with multiple branches

`cabal check` had a problem recognising fields in presence of
multiple branches. This patch fixes the problem and does not
meaningfully increases CI time of particularly taxing tests
(like “duplicate flagged dependencies” from MemoryUsage).

---------

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

This commit modifies the generation of runghc commands to pass an
additional -i argument to account for the change in working directory.

This ensures that, in the testsuite, runghc is able to see other modules.
For example, an invocation of runghc to run a Custom Setup script will
now properly see the modules imported, e.g. if one has a directory
structure like

  test.cabal
  Setup.hs
  SetupDep.hs

and Setup.hs imports SetupDep.hs.

Extra-packages listed in a cabal project are to be fetched from hackage,
and will be in memory as 'NamedPackages' rather than resolved to
'SpecificSourcePackage'.

On install, we need to make source-dists for all the specific source
packages, and fetch other packages from hackage. Since extra-packages
are already 'NamedPackages', we simply return them along the sdistize-d
specific source packages and the hackage source packages -- they will be
correctly fetched from Hackage from install.

Previously, cabal install <tgt> on a project with extra-packages would
fail because the branch of 'NamedPackage' for 'PackageSpecifier' was
simply unimplemented.

Fixes #8848

We must consider the path to the installed build-tool before the path to
existing versions of the build tool in paths such as `extra-prog-path`
or in the system path.

This was previously fixed by #8972 but undone by #9527.

This also renames `appendProgramSearchPath` to
`prependProgramSearchPath` to describe correctly what that function
does.

Fixes #9756

Not able to reproduce the problem this avoids. Tested with;

- --with-cabal=./dist-newstyle/build/x86_64-linux/ghc-9.6.3/cabal-install-3.11.0.0/x/cabal/build/cabal/cabal
--with-cabal=./dist-newstyle/build/x86_64-linux/ghc-9.8.1/cabal-install-3.11.0.0/x/cabal/build/cabal/cabal

The idea here is to pass a `--package-db` flag to `cabal-install` which
contains just `Cabal` and `Cabal-syntax` of the specific version. This
allows `cabal-install` tests to use the in-tree `Cabal` version,
something which you can easily run into and get very confused about when
writing tests.

There are a few options which can be passed to `cabal-tests` executable
to control which Cabal library you will test against.

1. --boot-cabal-lib specifies to use the Cabal library bundled with the
   test compiler, this is the default and existing behaviour of the
   testsuite.
2. --intree-cabal-lib=<root_dir> specifies to use Cabal and Cabal-syntax
   from a specific directory, and `--test-tmp` indicates where to put
   the package database they are built with.
3. --specific-cabal-lib=<VERSION> specifies to use a specific Cabal
   version from hackage (ie 3.10.2.0) and installs the package database
   into --test-tmp=<DIR>

The end result is that changes in the Cabal library can be tested with
cabal-install tests in the testsuite.

There have been a number of confusing issues with people writing tests
for changes in the Cabal library which never ran because of
cabal-install tests always used the boot Cabal library (see #9425
for one).

Fixes #9681

1. Working directory is not contaminated with results of the testsuite.
   Tests can be written in a more liberal way (creating/writing files
   etc).
2. The tests are more hermetic.. for example it was basically impossible
   to write a test which run outside of a project context.
3. The overall path lengths will be much shorter, which has been a
   consistent issue on windows. We can remove hacks like
   `withShorterPathForNewBuildStore`.

Fixes #9711

In this PR we make the `--package-db` flag apply only to the default
immutable initial package stack rather than also applying to the store
package database.

There are two special package databases which cabal install knows about
and treats specially.

* The store directory, a global cache of installed packages where cabal
  builds and installs packages into.
* The inplace directory, a local build folder for packages, where cabal
  builds local packages in.

It is very important that cabal registers packages it builds into one of
these two locations as there are many assumptions that packages will end
up here.

With the current design of the `--package-db` flag, you are apparently
allowed to override the store location which should have the effect of
making the last package database in the package stack the "store"
directory. Perhaps this works out in theory but practically this
behaviour was broken and things were always registered into the store
directory that cabal knew about. (The assertion in
`ProjectBuilding.UnpackedPackage` was failing (see added test)).

With `--package-db` not being able to modify the location of the store
directory then the interaction of `--package-db`, `--store-dir` and
`--dist-dir` flags become easy to understand.

* `--package-db` modify the initial package database stack, these
  package database will not be mutated by cabal and provide the initial
  package environment which is used by cabal.
* `--store-dir` modify the location of the store directory
* `--dist-dir` modify the location of the dist directory (and hence
  inplace package database)

Treating the flags in this way also fix an assertion failure when
building packages.

Fixes #9678

It fails in the old-ghcs 8.4.4 configuration, and debugging why seems to
have very little pay-off for this niche feature in a very old
configuration.

This patch deletes a lot of hairy logic to do with removing "bad" edges
from the closure. Now the solver just forces `Cabal >= 3.11` universally
when `--enable-multi-repl` is enabled.

Fixes #9710

This commit adds a Suffix newtype to describe suffixes as handled
by suffix handlers & preprocessors, and changes the PPSuffixHandler
type definition to use it.

It also moves some type definitions from Distribution.Simple.PreProcess
to the new module Distribution.Simple.PreProcess.Types.

As this commit changes the definition of PPSuffixHandler, it will
break custom Setup scripts which use the 'hookedPreProcessors'
functionality.

In #9697 we fixed passing local options to cabal install targets, but
rebasing accidentally dropped one of the added the tests for that PR,
which tested cabal install <target>, where <target> was only available
from a remote repository.

We use the datatype representation from the globbing in cabal-install,
but preserve a standalone parser for globs present in cabal files, whose
specification is constrained by the cabal specification. The
implementations are merged taking the best parts of each.

We also make sure sdist is robust to accidentally-listed directories,
as the refactor of the globbing modules re-uncovered this issue, and
required a prompt fix for the refactor not to break some things.

Fixes #5349

The target of `cabal install` is not considered to be a local package,
which means local configuration (e.g. in cabal.project, or flags like
--enable-profiling) does not apply to it.

In 76670ebd, we changed the behaviour to
applying the local flags to cabal install targets, but it used the
literal target string as a package name to which the flags were
additionally applied.

However, `cabal install` targets are NOT necessarily package names, so,
e.g., if we did `cabal install exe:mycomp`, the local flags would not
apply since "exe:mycomp" is not a recognized /package/.

The solution is to parse the target selectors first, and apply the local
flags to the package of the resolved targets.

Fixes #7297, #8909, the install part of #7236, #8529, #7832

Hannes Siebenhandl authored 1 year ago and Mikolaj Konarski committed 1 year ago

This reverts commit 3f4c81fd.

We were attempting to register a package into the global package
database, which then triggered some other check to do with relocatable
packages to fail.

The fix is to initialise and use a local package database, so the
package would be built and installed into the local package database.

- Allow for bad behaviour of master branch
- Add cyclical checks with same file names and hops
- Add cyclical import tests with 1 and 2 hops in cycle
- Expected output has project with full project path
- Add newlines at EOF