D347 introduced a bug wherein the event manager would drop registrations that
should be retained during processing. This occurs when an fd has multiple
registrations, not all of which fire, as well as the case of multi-shot
registrations.

I also do some general house-keeping, try to better document things, and fix a
bug which could result in unnecessary calls to `epoll_ctl`

Reviewed By: austin

Differential Revision: https://phabricator.haskell.org/D849

GHC Trac Issues: #10317