Implement -jsem: parallelism controlled by semaphores
See https://github.com/ghc-proposals/ghc-proposals/pull/540/ for a complete description for the motivation for this feature. The `-jsem` option allows a build tool to pass a semaphore to GHC which GHC can use in order to control how much parallelism it requests. GHC itself acts as a client in the GHC jobserver protocol. ``` GHC Jobserver Protocol ~~~~~~~~~~~~~~~~~~~~~~ This proposal introduces the GHC Jobserver Protocol. This protocol allows a server to dynamically invoke many instances of a client process, while restricting all of those instances to use no more than <n> capabilities. This is achieved by coordination over a system semaphore (either a POSIX semaphore [6]_ in the case of Linux and Darwin, or a Win32 semaphore [7]_ in the case of Windows platforms). There are two kinds of participants in the GHC Jobserver protocol: - The *jobserver* creates a system semaphore with a certain number of available tokens. Each time the jobserver wants to spawn a new jobclient subprocess, it **must** first acquire a single token from the semaphore, before spawning the subprocess. This token **must** be released once the subprocess terminates. Once work is finished, the jobserver **must** destroy the semaphore it created. - A *jobclient* is a subprocess spawned by the jobserver or another jobclient. Each jobclient starts with one available token (its *implicit token*, which was acquired by the parent which spawned it), and can request more tokens through the Jobserver Protocol by waiting on the semaphore. Each time a jobclient wants to spawn a new jobclient subprocess, it **must** pass on a single token to the child jobclient. This token can either be the jobclient's implicit token, or another token which the jobclient acquired from the semaphore. Each jobclient **must** release exactly as many tokens as it has acquired from the semaphore (this does not include the implicit tokens). ``` Build tools such as cabal act as jobservers in the protocol and are responsibile for correctly creating, cleaning up and managing the semaphore. Fixes #19349
parent
73d07c6e
Pipeline #64659 canceled
Stage: tool-lint
Stage: quick-build
Stage: full-build
Stage: packaging
Stage: testing
Showing
- .gitmodules 4 additions, 0 deletions.gitmodules
- cabal.project-reinstall 1 addition, 0 deletionscabal.project-reinstall
- compiler/GHC/Driver/Make.hs 72 additions, 48 deletionscompiler/GHC/Driver/Make.hs
- compiler/GHC/Driver/MakeSem.hs 548 additions, 0 deletionscompiler/GHC/Driver/MakeSem.hs
- compiler/GHC/Driver/Pipeline/LogQueue.hs 2 additions, 2 deletionscompiler/GHC/Driver/Pipeline/LogQueue.hs
- compiler/GHC/Driver/Session.hs 16 additions, 6 deletionscompiler/GHC/Driver/Session.hs
- compiler/ghc.cabal.in 2 additions, 0 deletionscompiler/ghc.cabal.in
- docs/users_guide/using.rst 53 additions, 0 deletionsdocs/users_guide/using.rst
- hadrian/src/Packages.hs 4 additions, 3 deletionshadrian/src/Packages.hs
- hadrian/src/Rules/ToolArgs.hs 1 addition, 0 deletionshadrian/src/Rules/ToolArgs.hs
- hadrian/src/Settings/Default.hs 2 additions, 0 deletionshadrian/src/Settings/Default.hs
- libraries/semaphore-compat 1 addition, 0 deletionslibraries/semaphore-compat
- packages 1 addition, 0 deletionspackages
Loading
Please register or sign in to comment