closeFdWith ought to be uninterruptible, since functions like hClose or Network.Socket.close are commonly used by bracket which only masks exceptions.
I explained here why closeFdWith can block (and thus interrupted by async exceptions), but I'll repeat the argument here for convenience.
closeFdWith is using takeMVar on two levels: it needs to access each EventManager, and for each of them it needs to get appropriate entry in the Array Int (MVar (IntTable [FdData])). A file descriptor is hashed and that hash is the entry in the array. The array has 32 entries, so hash collisions are expected.
Since this callback table might contain information about different file descriptors, it might be accessed concurrently by different threads, thus the inner takeMVar might block, although for a very short time as all the operations are using epoll (or a platform specific equivalent).
Since accessing the inner MVar (through callbackTableVar) might block, also accessing the MVar which holds EventManager can also block, but the only reason I can see why it would block is the one I gave above.
By making closeFdWith uninterruptible, we will guarantee that using hClose and Network.Socket.close is safe when used with bracket (interruptible close handler used in a bracket can leak a resource).