While looking at some preparatory work for !8073 I noticed that `GHC.Event.Thread.closeFdWith` is extremely suspicious. Specifically, it: 1. iterates over the capabilities and takes the event callback table `MVar` of each 2. iterates over the capabilities and calls `GHC.Event.Manager.closeFd_` on each previously-taken table to remove the to-be-closed fd 3. closes the fd 4. iterates over the capabilities and for each: * puts the event callback table `MVar` * runs the event notifier registered for the closed fd, if any All of this happens under an `uninterruptibleMask_`. First, a bit of background: `closeFdWith` is needed because POSIX does not specify the behavior of `select` or `poll` when an `fd` being blocked on in one thread is closed by another thread. Consequently, the IO manager must (atomically) ensure that all events registered against the fd are dispatched before the `fd` is closed. However, AFAICT, the above implementation can very easily break if the number of capabilities is concurrently changed. Specifically I can imagine the following problematic interleaving: ``` │ Capability 1 │ Capability 2 ──────────── │ ──────────── │ 0. `closeFdWith fd` is called │ │ 1. Takes event callback tables for all N │ capabilities │ │ │ 2. Calls `setNumCapabilities (N+1)`, blocks │ waiting to acquire all capabilities │ 3. Yields its capability (e.g. due to GC) │ │ │ 3. Takes capability, and now holds all capabilities. │ A new capability is spun-up and all capabilities │ are again released. │ │ 4. A thread is scheduled to capability N+1 │ │ 5. The thread registers an event for `fd` and blocks on `select`/`poll` │ 6. `closeFdWith` continues, calling │ `closeFd_` on N capabilities. │ This removes the fd from the event │ list of all capabilities and │ wakes up those that are blocked │ on the fd │ │ 7. the fd is closed │ │ │ 8. The thread is now blocked on `fd`, which was closed while │ the thread was blocked, triggering undefined behavior. │ ``` This is quite problematic.