Skip to content
  • sheaf's avatar
    Implement -jsem: parallelism controlled by semaphores · 5c873124
    sheaf authored and Marge Bot's avatar Marge Bot committed
    See https://github.com/ghc-proposals/ghc-proposals/pull/540/ for a
    complete description for the motivation for this feature.
    
    The `-jsem` option allows a build tool to pass a semaphore to GHC which
    GHC can use in order to control how much parallelism it requests.
    
    GHC itself acts as a client in the GHC jobserver protocol.
    
    ```
    GHC Jobserver Protocol
    ~~~~~~~~~~~~~~~~~~~~~~
    
    This proposal introduces the GHC Jobserver Protocol. This protocol allows
    a server to dynamically invoke many instances of a client process,
    while restricting all of those instances to use no more than <n> capabilities.
    This is achieved by coordination over a system semaphore (either a POSIX
    semaphore [6]_  in the case of Linux and Darwin, or a Win32 semaphore [7]_
    in the case of Windows platforms).
    
    There are two kinds of participants in the GHC Jobserver protocol:
    
    - The *jobserver* creates a system semaphore with a certain number of
      available tokens.
    
      Each time the jobserver wants to spawn a new jobclient subprocess, it **must**
      first acquire a single token from the semaphore, before spawning
      the subprocess. This token **must** be released once the subprocess terminates.
    
      Once work is finished, the jobserver **must** destroy the semaphore it created.
    
    - A *jobclient* is a subprocess spawned by the jobserver or another jobclient.
    
      Each jobclient starts with one available token (its *implicit token*,
      which was acquired by the parent which spawned it), and can request more
      tokens through the Jobserver Protocol by waiting on the semaphore.
    
      Each time a jobclient wants to spawn a new jobclient subprocess, it **must**
      pass on a single token to the child jobclient. This token can either be the
      jobclient's implicit token, or another token which the jobclient acquired
      from the semaphore.
    
      Each jobclient **must** release exactly as many tokens as it has acquired from
      the semaphore (this does not include the implicit tokens).
    ```
    
    Build tools such as cabal act as jobservers in the protocol and are
    responsibile for correctly creating, cleaning up and managing the
    semaphore.
    
    Adds a new submodule (semaphore-compat) for managing and interacting
    with semaphores in a cross-platform way.
    
    Fixes #19349
    5c873124