getNumCapabilities produces incorrect CPU counts
To reproduce this bug, use the following main.hs
:
import Control.Concurrent
main =
print =<< getNumCapabilities
Compile it with ghc -O2 -threaded --make main.hs -o reproduce
and run ./reproduce
in a virtualized environment such as Travis CI.
- *Expected output:** it prints the number of CPU cores available to the process, which in the case of Travis's virtualized environment is 2.
- *Actual output:** it prints the number of *physical* cores on the machine, which in the case of Travis's servers is 32.
Example of Failure on Travis
Here is an example build on Travis which demonstrates this failing: (scroll to the end of the console log to see the failure) https://travis-ci.org/rtfeldman/repro-ghc-bug/builds/340268785
Here is the repository which was used to reproduce this failure on Travis: https://github.com/rtfeldman/repro-ghc-bug
Real-World Consequences
The real-world consequences of this bug manifest for the compiler for the Elm programming language, which is written in Haskell and compiled with GHC. By default, elm-make
runs extremely slowly on Travis and Circle CI because it's trying to parallelize across 32 cores when only 2 are actually available to it.
How to Fix
For an example implementation which correctly detects the number of available cores (as opposed to physical), here is the source code to a Rust library which does so correctly: https://github.com/seanmonstar/num_cpus/blob/master/src/lib.rs - the library distinguishes between "number of CPUs" and "number of physical CPUs," and on Travis it correctly reports 2 for CPUs and 32 for physical.