sheaf authored 2 years ago and Marge Bot committed 1 year ago

See https://github.com/ghc-proposals/ghc-proposals/pull/540/ for a
complete description for the motivation for this feature.

The `-jsem` option allows a build tool to pass a semaphore to GHC which
GHC can use in order to control how much parallelism it requests.

GHC itself acts as a client in the GHC jobserver protocol.

```
GHC Jobserver Protocol
~~~~~~~~~~~~~~~~~~~~~~

This proposal introduces the GHC Jobserver Protocol. This protocol allows
a server to dynamically invoke many instances of a client process,
while restricting all of those instances to use no more than <n> capabilities.
This is achieved by coordination over a system semaphore (either a POSIX
semaphore [6]_  in the case of Linux and Darwin, or a Win32 semaphore [7]_
in the case of Windows platforms).

There are two kinds of participants in the GHC Jobserver protocol:

- The *jobserver* creates a system semaphore with a certain number of
  available tokens.

  Each time the jobserver wants to spawn a new jobclient subprocess, it **must**
  first acquire a single token from the semaphore, before spawning
  the subprocess. This token **must** be released once the subprocess terminates.

  Once work is finished, the jobserver **must** destroy the semaphore it created.

- A *jobclient* is a subprocess spawned by the jobserver or another jobclient.

  Each jobclient starts with one available token (its *implicit token*,
  which was acquired by the parent which spawned it), and can request more
  tokens through the Jobserver Protocol by waiting on the semaphore.

  Each time a jobclient wants to spawn a new jobclient subprocess, it **must**
  pass on a single token to the child jobclient. This token can either be the
  jobclient's implicit token, or another token which the jobclient acquired
  from the semaphore.

  Each jobclient **must** release exactly as many tokens as it has acquired from
  the semaphore (this does not include the implicit tokens).
```

Build tools such as cabal act as jobservers in the protocol and are
responsibile for correctly creating, cleaning up and managing the
semaphore.

Adds a new submodule (semaphore-compat) for managing and interacting
with semaphores in a cross-platform way.

Fixes #19349

Sylvain Henry authored 2 years ago and Marge Bot committed 2 years ago

`libiserv` serves no purpose. As it depends on `ghci` and doesn't have
more dependencies than the `ghci` package, its code could live in the
`ghci` package too.

This commit also moves most of the code from the `iserv` program into
the `ghci` package as well so that it can be reused. This is especially
useful for the implementation of TH for the JS backend (#22261, !9779).

Torsten Schmits authored 2 years ago and Marge Bot committed 2 years ago

Part of proposal 475 (https://github.com/ghc-proposals/ghc-proposals/blob/master/proposals/0475-tuple-syntax.rst)

Moves all tuples to GHC.Tuple.Prim
Updates ghc-prim version (and bumps bounds in dependents)

updates haddock submodule
updates deepseq submodule
updates text submodule

Haskeline now depends upon exceptions. See #16752.

Ryan Scott authored 6 years ago and Marge Bot committed 6 years ago

This moves all URL references to Trac tickets to their corresponding
GitLab counterparts.

Alec Theriault authored 6 years ago and Marge Bot committed 6 years ago

`parallel` is used in exactly one place in the GHC tree: the T2317 test.
It seems almost by accident that it is a submodule; libraries needed
only for tests should net be included as submodules (see `QuickCheck`,
`async`, `haskell98`, `regex-compat`, `utf8-string`, `vector` and more
for examples).

T2317 will now get run only when `parallel` is installed instead of
`parallel` being required for the testsuite to run.

I believe this was originally introduced to help test DPH, which is now
gone.

Previously, the `make clean` (as well as `make dist-clean`) doesn't work
for ghc-heap and libiserv, due to these two libraries are not presented
in the "packages" file.

Test Plan: [skip ci]

Reviewers: bgamari

Reviewed By: bgamari

Subscribers: rwbarton, thomie, carter

Differential Revision: https://phabricator.haskell.org/D4878

Ben Gamari authored 6 years ago and Ben Gamari committed 6 years ago

Poor DPH and its vectoriser have long been languishing; sadly it seems there is
little chance that the effort will be rekindled. Every few years we discuss
what to do with this mass of code and at least once we have agreed that it
should be archived on a branch and removed from `master`. Here we do just that,
eliminating heaps of dead code in the process.

Here we drop the ParallelArrays extension, the vectoriser, and the `vector` and
`primitive` submodules.

Test Plan: Validate

Reviewers: simonpj, simonmar, hvr, goldfire, alanz

Reviewed By: simonmar

Subscribers: goldfire, rwbarton, thomie, mpickering, carter

Differential Revision: https://phabricator.haskell.org/D4761

[skip ci]

This is a preparation for `haskeline` picking up a dependency on `stm`
real soon now. See https://github.com/judah/haskeline/pull/61 for details.

If we figure out a way to not bundle the libraries depended upon by the
GHCi executable in the global package database (see #8919 for the original
reason why we had to start bundling terminfo/haskeline in the first place)
we can get rid of `stm` again...

On the bright side, we were able to avoid uploading new `stm` releases for
over two years already, so it shouldn't cause too much trouble if GHC imposes
a strong preference on the `stm` package's version (this most likely will
mostly affect Linux distributions & similiar).

While at it, this also update the stm submodule to include relaxed
bounds to allow the upcoming base-4.11 version.

These three submodules have commits which add certain files to their
respective .gitignores which GHC's build system produces. Also update
the packages file accordingly.

Test Plan: If it builds, ship it

Reviewers: hvr, austin, bgamari

Subscribers: rwbarton, thomie

Differential Revision: https://phabricator.haskell.org/D3842

Herbert Valerio Riedel authored 7 years ago and Ben Gamari committed 7 years ago

Cabal's parser has been rewritten in terms of Parsec (which is not
enabled yet in Cabal-2.0 by default, but can be enabled by a cabal
flag). The plan for Cabal is to drop support for the non-parsec parser,
so we need to prepare GHC to cope with new situation.

However, this means that lib:Cabal requires three new library
dependency submodules,

 - parsec
 - text
 - mtl

What complicates matters is that we need to build `ghc-cabal` early on
during the bootstrap phase which currently needs to invoke `ghc --make`
directly. So these additional dependencies need to be integrated into
the monolithic `ghc --make` invocation which produces the `ghc-cabal`
executable.

Test Plan: `./validate --fast` passed

Reviewers: austin, bgamari

Subscribers: erikd, phadej, rwbarton, thomie

Differential Revision: https://phabricator.haskell.org/D3757

Michal Terepeta authored 7 years ago and Ben Gamari committed 7 years ago

This copies the subset of Hoopl's functionality needed by GHC to
`cmm/Hoopl` and removes the dependency on the Hoopl package.

The main motivation for this change is the confusing/noisy interface
between GHC and Hoopl:
- Hoopl has `Label` which is GHC's `BlockId` but different than
  GHC's `CLabel`
- Hoopl has `Unique` which is different than GHC's `Unique`
- Hoopl has `Unique{Map,Set}` which are different than GHC's
  `Uniq{FM,Set}`
- GHC has its own specialized copy of `Dataflow`, so `cmm/Hoopl` is
  needed just to filter the exposed functions (filter out some of the
  Hoopl's and add the GHC ones)
With this change, we'll be able to simplify this significantly.
It'll also be much easier to do invasive changes (Hoopl is a public
package on Hackage with users that depend on the current behavior)

This should introduce no changes in functionality - it merely
copies the relevant code.

Signed-off-by: Michal Terepeta <michal.terepeta@gmail.com>

Test Plan: ./validate

Reviewers: austin, bgamari, simonmar

Reviewed By: bgamari, simonmar

Subscribers: simonpj, kavon, rwbarton, thomie

Differential Revision: https://phabricator.haskell.org/D3616

This reverts commit 8d4bce42.

This is rather annoying. I'd prefer to have a stable release to
use. However libffi-3.2.1 has been released November 12, 2014, and
libffi-4 is TBD. See also https://github.com/libffi/libffi/issues/296

The core reason for this change is that llvm changed the supported
assembly to unified syntax, which libffi-3.2.1 does not use, and hence
fails to compile for arm with llvm. For refence, see the following
issue: https://github.com/libffi/libffi/issues/191

Reviewers: austin, bgamari

Reviewed By: bgamari

Subscribers: rwbarton, thomie, snowleopard

Differential Revision: https://phabricator.haskell.org/D3349

Summary:
The plan is to release a separate library, 'compact', which gives a
friendly user-facing interface.  This library is just enough so that we
can make sure the functionality is working in GHC.

Signed-off-by: Edward Z. Yang <ezyang@cs.stanford.edu>

Test Plan: validate

Reviewers: bgamari, dfeuer, austin, simonmar, hvr

Subscribers: thomie, erikd, snowleopard

Differential Revision: https://phabricator.haskell.org/D3206

dobenour authored 8 years ago and Ben Gamari committed 8 years ago

Reviewers: bgamari, austin

Reviewed By: bgamari

Subscribers: thomie

Differential Revision: https://phabricator.haskell.org/D2733

Signed-off-by: Edward Z. Yang <ezyang@cs.stanford.edu>

It will get built as part of "validate --slow".  We have 11 tests that
depend on random, so this enables more tests to work.

Effective immediately, pushing to the `libraries/parallel` submodule
requires pushing via ssh://git@github.com/haskell/parallel.git.

Summary:
As discussed on ghc-devs@haskell.org and the trac ticket, we're removing
these packages from the 7.10 release as they no longer work correctly,
and can't easily be made to properly follow the standard as `base`
changes over time.

This does not remove the packages from the tree, only the build system.

https://www.haskell.org/pipermail/ghc-devs/2014-November/007357.html



Signed-off-by: Austin Seipp <austin@well-typed.com>

Test Plan: iiam

Reviewers: hvr, ekmett

Reviewed By: hvr, ekmett

Subscribers: thomie, carter

Differential Revision: https://phabricator.haskell.org/D510

GHC Trac Issues: #9590

Effective immediately, `deepseq` is maintained officially by the
core-library-comittee.  Moreover, pushing to libraries/deepseq requires
pushing to ssh://git@github.com/haskell/deepseq.git from now on.

[skip ci]

Effective immediately, pushing to libraries/filepath requires
pushing to ssh://git@github.com/haskell/filepath.git.

Effective immediately, pushing to libraries/{directory,process} requires
pushing to ssh://git@github.com/haskell/{directory.process}.git. This has
been done now even though there's no scripted tooling yet as the GitHub repo
was already getting issues filed.

The `time` package has recently moved its primary upstream location
to https://github.com/haskell/time.

This leaves `transformers` the only package used by GHC still requiring
darcs2git mirroring machinery.

Effective immediately, pushing to libraries/unix requires pushing to
ssh://git@github.com/haskell/unix.git. This done now even though there's
no scripted tooling yet as the GitHub repo is already receiving pull requests
(which are getting merged)

Specifically, the following sub-repos/modules are converted:

 - libffi-tarballs
 - libraries/array
 - libraries/deepseq
 - libraries/directory
 - libraries/dph
 - libraries/filepath
 - libraries/haskell2010
 - libraries/haskell98
 - libraries/hoopl
 - libraries/hpc
 - libraries/old-locale
 - libraries/old-time
 - libraries/parallel
 - libraries/process
 - libraries/stm
 - libraries/unix
 - nofib
 - utils/hsc2hs

N.B. ghc-tarballs is not converted as it will probably be handled
     differently in the future.

Signed-off-by: Herbert Valerio Riedel <hvr@gnu.org>

In the future, some script will use the last column to configure
Git to redirect `push` operations to the respective primary Git
repo.

One way could be via the `pushInsteadOf` facility described in

  https://ghc.haskell.org/trac/ghc/wiki/WorkingConventions/Git/Submodules



Signed-off-by: Herbert Valerio Riedel <hvr@gnu.org>

This adapts `sync-all` and `packages` to the recently folded-in Git
repositories

 - `base.git`
 - `ghc-prim.git`
 - `integer-gmp.git`
 - `integer-simple.git`
 - `template-haskell.git`

See #8545 as well as 5f54d678 for more
details.

Signed-off-by: Herbert Valerio Riedel <hvr@gnu.org>

This should help contribute content to

  https://ghc.haskell.org/trac/ghc/wiki/WorkingConventions/Git/Submodules



Signed-off-by: Herbert Valerio Riedel <hvr@gnu.org>

See merge commit 66693401

This commit also adds a check for a left-over testsuite/.git folder to sync-all

This way, the first time sync-all is called after updating to a
post-testsuite-merge (see #8545) state of ghc.git, the sync-all script
aborts with an error message if a `testsuite/.git` folder is detected and
thus forces the user to take action.

Signed-off-by: Herbert Valerio Riedel <hvr@gnu.org>

Signed-off-by: Herbert Valerio Riedel <hvr@gnu.org>

Thus GHC's in-tree version corresponds exactly to
[hackage:terminfo-0.3.2.6]. Morever, `terminfo` has moved
to GitHub as well (https://github.com/judah/terminfo

) so
the upstream repo URL in `packages` is updated as well.

Signed-off-by: Herbert Valerio Riedel <hvr@gnu.org>

[hackage:haskeline] has now an official GitHub repo at
https://github.com/judah/haskeline.git



Consequently, there remain only 3 repos which are Darcs hosted
(and thus still require darcs2git mirroring), namely
[hackage:time], [hackage:terminfo], and [hackage:transformers].

Signed-off-by: Herbert Valerio Riedel <hvr@gnu.org>

Signed-off-by: Herbert Valerio Riedel <hvr@gnu.org>

 - `primitive` is updated to upstream's HEAD which is essentially
   `primitive-0.5.1.0`, plus a core-lint-error workaround for #8355 and
   some minor cleanups.

 - `vector` is updated to upstreams `vector-0.10.9.1` release

Note: The upstream repo location has changed to GitHub,
      hence the update in the `packages` file

Signed-off-by: Herbert Valerio Riedel <hvr@gnu.org>

By keeping the repository url in the `packages` file scripts such as
source:ghc/sync-all can pick it up. Moreover, it's easier to
automatically validate for correctness than the
wiki:Repositories/Upstream page.

This means that ghc-tarballs is only needed on Windows