In the 2decb0e7 refactor we stopped
looking for non-Haskell `main-is` files in the hs-source-dirs and other
appropriate directories. This commit fixes that oversight.

Even if it is not intuitive that main-is-C-sources are searched in the
hs-source-dirs, we don't wish to break users relying on this behaviour
as there does not exist that strong of a motivation to do so.

Fixes #10168

Co-authored-by: sheaf <sam.derbyshire@gmail.com>

This commit makes haddock-project handle sublibraries.

The commit changes how `cabal haddock` works, changing the layout
in the `dist-newstyle` folder.  With this change `haddock` subcommand
will install `package:sublib` component in a directory `package-sublib`
under `l/sublib/doc/html/`.

Add a flag to disable the hard requirement on the
build-tools-(depends) declared on the Cabal package.

When this flag is enabled (--ignore-build-tools), a build-tool which
can't be found does not block compilation.

Fixes #10061

When no pkg-config program was found, cabal would
claim that the package is not in the db, instead
of telling clearly that no pkg-config was found at
all.

New options for cabal.project and ./Setup interface:

* `profiling-shared`: Enable building profiling dynamic way
* Passing `--enable-profiling` and `--enable-executable-dynamic` builds
  profiled dynamic executables.

Support for using `profiling-shared` is guarded behind a constraint
which ensures you are using `Cabal >= 3.13`.

In the cabal file:

* `ghc-prof-shared-options`, for passing options when building in
  profiling dynamic way

Other miscellenious fixes and improvements

* Some refactoring around ways so that which
  ways we should build for a library, foreign library and executable is
  computed by the `buildWays` function (rather than ad-hoc in three
  different places).

* Improved logic for detecting whether a compiler supports compiling
  a specific way. See functions `profilingVanillaSupported`,
  `dynamicSupported`, `profilingDynamicSupported` etc
  These functions report accurate infomation after ghc-9.10.1.

* Fixed logic for determining whether to build shared libraries. (see
  #10050)
  Now, if you explicitly enable `--*-shared`, then that will always take
  effect. If it's not specified then `--enable-executable-dynamic` will
  turn on shared libraries IF `--enable-profiling` is not enabled.

* Remove assumption that dynamically linked compilers can build dynamic
  libraries (they might be cross compilers.

* Query the build compiler to determine which library way is necessary
  to be built for TH support to work.
  (rather than assume all compilers are dynamically linked)

* An extensive test which checks how options for `./Setup` and
  `cabal-install` are translated into build ways.

Fixes #4816, #10049, #10050

We should configure unconfigured programs before serialising them
(using the Binary ProgramDb instance) in
Distribution.Client.ProjectPlanning.configureCompiler, as otherwise
we can accidentally discard information.

This isn't currently a live bug, because in practice we end up
re-running configuration steps when interfacing with the Cabal library.
However, in the bright future in which we avoid re-configuring things
when building a Cabal package that we already determined when invoking
cabal-install, this starts causing problems.

Be sure to invalidate the cache if building throws an exception!
If not, we'll abort execution with a stale recompilation cache.
See ghc#24926 for an example of how this can go wrong.

Issue #10063 points out that cabal exits before the external command
has finished executing. This was a simple oversight to not
waitForProcess on the result of calling createProcess.

This also points out the flaw that there isn't a way for external
commands to signal failure, so we now also propagate the exit code from
the external process.

Fixes #10063

Currently the options `--program-{prefix,suffix}` for cabal install
affects the name of the file in the install directory *and* the
executable name in the store. The installation fails:

- If using `--install-method=symlink`, the *target* of the symlink is
  not affected by the affix options and it results in an invalid symlink.
- If using `--install-method=copy`, the copy fails because the source
  is not found.

Another issue is that it affects the computation of the hash of the
build directory in the store, resulting in needless rebuild when using
successively different affix options.

Fixed by making the name of the executable in the store canonical, i.e.
always ignoring the program affix options.

Added a test for all the combinations of `--install-method` and program
affixes options.

These options are configured into the program by Cabal, so we also need
to extract these and put them into the mutli-file.

Fixes #10015

Haddock no longer writes compilation files by default, so we do not need to pass
tmp dirs for `-hidir`, `-stubdir`, and `-odir` via `--optghc`. Indeed, we do not
*want* to do so, since it results in recompilation for every invocation of
Haddock via Cabal. This commit stops this from happening for haddock versions >=
2.28 (when Hi Haddock was introduced).

This commit also stops the default definition of the `__HADDOCK_VERSION__` macro
when invoking GHC through haddock. Since a very limited set of users may still
depend on this macro, we introduce the `--haddock-version-cpp` flag and
`haddock-version-cpp:` cabal.project field, which enable the definition of
the `__HADDOCK_VERSION__` macro when invoking GHC through Haddock. This will
almost guarantee recompilation during documentation generation due to the macro
definition.

This commit also renames the `--haddock-lib` flag to `--haddock-resources-dir`
(and `haddock-lib:` cabal.project field to `haddock-resources-dir:`), and adds
this flag to the users guide since it was missing an entry. This also allows us
to add this field to `cabal-install:test:integration-tests2`, since it is no
longer ambiguous with the `--lib` flag.

This commit also causes `documentation: true` or `--enable-documentation` to
imply `-haddock` for GHC.

Also, since Haddock >= 2.29 is renaming `--lib` to `--resources-dir`, this
commit switches the flag provided to Haddock using a backwards compatible
condition based on the Haddock version.

Adds a changelog entry.

* CI: add GHC 9.10

* GHC 9.10 compat in testsuite: CPP symbols don't get passed to CC as
eagearly

Discussion: https://github.com/haskell/cabal/pull/9914#issuecomment-2077439625

Related GHC issue: ghc/ghc#21291

* GHC 9.10 compat in tests: disable tests regressing due to #9940

Disallowing whitespace while parsing target selectors incorrectly
disallows file targets with whitespace in the paths, which can issues
when users pass absolute paths.

fixes #8875

RejectFutureIndexStates and UpdateIndexState are marked "skip", but it's
under withRemoteRepo, which causes flakiness before skip is called.

This commit makes Location a separate data type:

  data Location where
    Location
      :: SymbolicPath Pkg (Dir baseDir)
      -> RelativePath baseDir File
      -> Location

instead of being a type synonym for (FilePath, FilePath).

We noted during testing of the Hooks API that it was all too easy to
give an incorrect location for rule outputs, e.g. by omitting an
extension or using an absolute path.
This change allows us to improve the API documentation, as well as
clarifying the types to avoid any ambiguities about what kind of file
path is expected (FilePath vs SymbolicPath).

- Use real-life example from Cabal project in doctest

This allows us to compile Setup.hs without depending on base.
In particular, this ensures that a package with `build-type: Hooks`
and a custom setup stanza that does not depend on base successfully
compiles.

Tested in PackageTests/SetupHooks/SetupHooksNoBase.

This commit initialises the Haddock flag haddockKeepTempFiles with the
value of buildSettingKeepTempFiles.

This addresses an 8 year old TODO in the code, from Duncan Coutts.

This makes Setup copy/install succeed if there's
nothing to do because the package doesn't contain
a library or executable.

This allows downstream users of Cabal to avoid having to
add workarounds for this edge case.

Resolves #6750

Previously `--package-db` was only passed to test which used the `v2-`
prefix.

Now we pass `--package-db` to things which use the `v2-` prefix, not
things which use the `v1-` prefix and by default assume that unprefixed
commands are v2 commands.

The --working-dir flag is only available for Cabal >= 3.13.
This commit fixes an incorrect conditional: we only filtered out this
flag for Cabal < 3.11, instead of < 3.13.

Test: PackageTests/WorkingDir

Fixes #9940

This patch ensures that we correctly provision executables declared
in the build-tool-depends fields in all circumstances:

  - whether the build tool is external (from another package) or
    internal (declared in the current package)
  - whether the build tool is used at compile time (e.g. in a pre-build
    rule or in a Template Haskell splice) or at run time (e.g. when
    running a test-suite, benchmark or executable).

Note that correctly provisioning a build tool requires two pieces of
information:

  - making it available in PATH,
  - ensuring it has the correct environment variables overrides;
    in particular, the build tool needs to be able to find its own
    data directory.

The test case BuildToolPaths checks all of these situations are handled
correctly.

Matthew Pickering authored 1 year ago and Rodrigo Mesquita committed 11 months ago

This makes `withShorterPathForNewBuildStore` fit more nicely into the
rest of the testing infrastructure.

* Move `withShorterPathForNewBuildStore` to `TestM` monad
* Move responsibility for passing `--store-dir` to `cabalGArgs` function
* Move `findDependencyInStore` into `TestM`, and remove requirement to
  pass path to store directory.
* Introduce `testStoreDir` function which returns the store location
  (and honours `withShorterPathForNewBuildStore`)
* Migrate tests which use `withShorterPathForNewBuildStore`.

This commit introduces a new build-type, Hooks. A package using this
build type should provide a SetupHooks.hs module which exports
a value `setupHooks :: SetupHooks`. This is intended to replace the
Custom setup type. This allows Cabal to have finer-grained information
about the build, instead of having an opaque Setup executable to invoke.

Tests include:

  - error when returning an invalid component diff in a
    per-component pre-configure hook
  - error when declaring pre-build rules whose dependency
    graph contains cycles
  - error when we cannot find a dependency of a pre-build rule
  - warning when there are pre-build rules that are declared but
    never demanded
  - correctness tests for SetupHooks, e.g. that
    pre-build rules are run in dependency order (see the
    `SetupHooksRuleOrdering` test)

Add tests for #9799 about freeze qualifying all packages with 'any'
constraint scope.

This reverts commit a90d44ff.

* Redesign 'cabal path' command to account for projects

Previously, `cabal path` was only able to query from the global
configuration file, e.g., `~/.cabal/config` or the XDG equivalent.
Adding support for cabal project is a huge boost to usability.

We take the foundations and turn them into `cabal v2-path` which takes
project configuration, such as `cabal.project` into account.
Note, the command is still named `cabal path`, but for the sake of
disambiguation, we refer to this new iteration of the command as `cabal
v2-path`.

In addition, we add support for multiple output formats, such as
key-value pairs and json.

The key-value pair output prints a line for each queried key and its
respective value:

    key1: value2
    key2: value2

If only a single key is queried, we print only the
value, for example:

    value1

The json output format is versioned by the cabal-install version which
is part of the json object.
Thus, all result objects contain at least the key "cabal-install-version".

We expand the `cabal v2-path` to also produce information of the
compiler that is going to be used in a `cabal build` or `cabal repl` invocation.
To do that, we rebuild the install plan and query for the configured
compiler program.
This is helpful for downstream tools, such as HLS, to figure out the GHC
version required to compile a project with.

We also add an exhaustive test suite for 'cabal path' cmd

We test that each query honours cabal.project files, cli parameters, and
is composable with the other query flags.

We extend the test output normalisers for ghc compiler location and
cabal-install version, as the 'cabal path' command outputs the exact ghc
and ghc-pkg location. In addition, the json output format is versioned
on the cabal-install version.

Currently, we query the cabal-install version on each test normalisation
run. This might be unnecessary expensive, and could be avoided by
introducing a 'cabalProgram' that specifies how the program version can
be found. This way, we can cache the version query.

Add '--cache-home' flag thats shows the cabal's cache root

Rename '--cache-dir' to the more correct '--remote-repo-dir'.

* Update 'cabal path' documentation

* Add changelog.d entry

---------

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

With this change to the solver message rendering, I also fix some bugs around project imports, adding tests for those cases. Reviewers asked that the Y-shaped import checks (using IORef) be made on a separate pull request. Removing those lead to cascading deletions.

- Regenerate expected .out files
- Show tree provenance of import constraint
- Add trimmed down PackageTests/VersionPriority
- Add changelog entry
- Use NonEmpty
- Fix check for cyclical import
- Use primes for next iteration
- Remove unused LANGUAGE pragmas
- Rename to projectConfigPathRoot
- Docs for ProjectConfigPath and showProjectConfigPath
- Renaming
- Add cyclical import tests with 1 and 2 hops in cycle
- Use full path for cyclical error message
- Expected output has project with full project path
- Add fullPath local function
- Project directory as FilePath, not Maybe FilePath
- Use (_, projectFileName) binding splitFileName
- Need full path to project parsing legacy
- Inline seenImports conversion
- Add cyclical checks with same file names and hops
- Add noncyclical tests that hop over folders
- Add a project testing skipping in and out of a folder
- Update expectations of cyclical tests
- Use canonicalizePath for collapsing .. when possible
- Capture trace for later
- Add module for ProjectConfigPath
- Move functions for ProjectConfigPath to its module
- Fetch URI is not prefixed with ./https://etc
- Document normaliseConfigPath
- Add doctests for normaliseConfigPath
- Add doctest of canonicalizeConfigPath
- Show an example of canonical paths
- Use importer and importee in canonicalizeConfigPaths
- Add logging
- Use normLocPath, drop -XMultiWayIf
- Remove seenImports parameter from parseProject
- Follow hlint suggestion: Move brackets to avoid $
- Follow hlint warning: Use mapM
- Follow hlint warning: Use traverse_
- Follow hlint suggestion: Use <$>
- Add failing test skipping across folder
- Remove normaliseConfigPath
- Add hasDuplicatesConfigPath
- Remove lengthConfigPath and nubConfigPath
- Mention cabal-install-solver in the changelog
- Left align project import tree with other content
- Rerun tests to generate left-aligned tree output
- Don't show "constraint from project requires ..."
- Regenerate test outputs without (constraint from project ...)
- Move fail reason printing to project config path module
- Minimize diff of showFR
- Show an "imported by: " list instead of a tree
- Update info logging, avoiding leading spaces
- When logging I found leading spaces were trimmed
- Don't log relative location and path
- Use unicode tree symbol in haddocks
- A duplicate importing tests
- Add Y-forking import test
- A test for detecting when the same config is imported via many different paths
- Fix a typo with ../noncyclical-same-filename-b.config
- Add an intercepting cabal-testsuite/cabal.project
- Use IORef to detect seen imports
- Get rid of seenImports
- Use IORef [FilePath] of uniqueImports
- Add consProjectConfigPath
- Expand the haddocks of newtype ProjectConfigPath
- Clarify, it's the project root directory
- Rename a function to makeRelativeConfigPath
- Fix whitespace
- Update output snippet in changelog
- Put the info logging together
- Split cycles and duplicates detection
- Add fixes and duplicate import to the change log
- Better reporting of duplicate imports
- Report cycles and duplicates using unique file name
- This is after canonicalizeConfigPath, relative to the project directory
- Get rid of fullLocPath, use normLocPath only
- Do reporting using paths relative to the directory of the project
- Issue a warning for Y-shaped duplicate imports
- Add duplicateImportMsg
- Use pretty printing for duplicate import message
- Use pretty printing for all ProjectConfigPath printing
- Drop a parameter from duplicateImportMsg
- Correct haddocks on docProjectConfigPath
- Satisfy fourmolu
- Remove the surplus space in "imported by "
- Log project parsing at the debug level, up from info
- Simplify the changelog snippet
- Use normalized paths for solver messages
- Satisfy fourmolu
- Relative to the directory of the project
- Duplicate imports are reported as warnings
- No need to mention v2-build
- Remove IORef
- Subdue a check for duplicate imports
- TODO: Catch duplicates across different import paths. We know how to do this
- Remove check for duplicate imports
- Improve the canonicalizeConfigPath doctests
- Add haddocks, drop configPath from go
- Make importsBy NonEmpty
- Use testDir in doctests
- Use canonical test directory
- Regenerate expected outputs
- Assume head of the path is the duplicate
- Get rid of duplicateImportMsg
- Remove unused imports in ProjectConfigPath
- Follow hlint suggestion: use fewer imports
- Remove fullConfigPathRoot
- Follow hlint suggestion: redundant bracket
- Use the original "source" variable name
- Put the source next to its contents
- Go with the original "seenImports" variable name
- Avoid -XMultiWayIf
- Not needed since we aren't checking for duplicates right now
- Be consistent in naming sources
- Move project dir alongside related params
- Rename to pathRequiresVersion
- Add back "constraint from"
- Talk only of cycles and not duplicates
- Don't report line number for cyclical import
- Remove 2nd example in changelog
- Remove Y-forked duplicate import test
- Add "imported by" to assertOutputContains
- Cull excess doctests of canonicalizeConfigPath
- Add deep path doctest
- Update haddocks of canonicalizeConfigPath
- remove idempotent speculation (it isn't idempotent)
- use "indirection" to describe "." and ".."
- makes the path relative to the given directory
- Remove seenImports
- Use Explicit ProjectConfigPath
- Missed one "constraint from" in changelog example
- Don't check (verbosity >= verbose) separately
- While the same level of verbosity as info, it is clearer to not check the level separately.
- Might calculate the path but not use it.
- Add AbsoluteDir
- Keep provenance normalised for display
- Remove stale REVIEW comment
- Storing provenance normalized avoids this
- Fix whitespace
- Satisfy HLint warning: Redundant flip
- Gut the repo/hashable packages
- Revert AbsoluteDir to FilePath

This commit makes the library functions in Cabal agnostic of the working
directory. In practice, this means that we distinguish `FilePath`s
from un-interpreted `SymbolicPath`s. The latter may be paths that are
relative to the working directory, and need to be interpreted with
respect to a passed-in argument specifying the working directory,
instead of using the working directory of the current process.
See Note [Symbolic paths] in Distribution.Utils.Path.

In particular, paths in the package description now use the SymbolicPath
abstraction, which allows specifying whether they are allowed to be
absolute, and, if they are relative, what they are relative to.
For example, source files are relative to a source search directory,
data files are relative to the data directory, and doc files are
relative to the package root.

Fixes #9702

As a warning (“Hackage would reject this package”), to prepare users
to GHC language editions.

- Fix whitespace in cabal-testsuite + other tests

* Add tests for #9742

`main-is` not picked up when inside a multibranch CondNode.

* Fix comments

* Add simplifyBranch to Distribution.Types.CondTree

Goes hand in hand with with simplifyCondTree.

* Make `check` deal correctly with multiple branches

`cabal check` had a problem recognising fields in presence of
multiple branches. This patch fixes the problem and does not
meaningfully increases CI time of particularly taxing tests
(like “duplicate flagged dependencies” from MemoryUsage).

---------

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

This commit modifies the generation of runghc commands to pass an
additional -i argument to account for the change in working directory.

This ensures that, in the testsuite, runghc is able to see other modules.
For example, an invocation of runghc to run a Custom Setup script will
now properly see the modules imported, e.g. if one has a directory
structure like

  test.cabal
  Setup.hs
  SetupDep.hs

and Setup.hs imports SetupDep.hs.

Extra-packages listed in a cabal project are to be fetched from hackage,
and will be in memory as 'NamedPackages' rather than resolved to
'SpecificSourcePackage'.

On install, we need to make source-dists for all the specific source
packages, and fetch other packages from hackage. Since extra-packages
are already 'NamedPackages', we simply return them along the sdistize-d
specific source packages and the hackage source packages -- they will be
correctly fetched from Hackage from install.

Previously, cabal install <tgt> on a project with extra-packages would
fail because the branch of 'NamedPackage' for 'PackageSpecifier' was
simply unimplemented.

Fixes #8848

We must consider the path to the installed build-tool before the path to
existing versions of the build tool in paths such as `extra-prog-path`
or in the system path.

This was previously fixed by #8972 but undone by #9527.

This also renames `appendProgramSearchPath` to
`prependProgramSearchPath` to describe correctly what that function
does.

Fixes #9756