System.Exit.exitWith exits current thread, not program
The documentation for System.Exit is a bit ambiguous about what the exitWith function exits from, but IMHO the most natural inference is that it exits the process from within which it is called. However, it currently does not necessarily do so; if called from a forkIO'ed thread, it will raise the exit exception in that thread rather than the "main thread" of the process.
IMHO it would be best to change this behavior, but failing that it ought to at least be better documented. For now, I am manually throwing an !ExitCode to the main thread as a workaround.