GitLab

Clean up code&interfaces that deals with timers and asynchrony:

- Timer.{c,h} now defines the platform-independent interface
  to the timing services needed by the RTS. Itimer.{c,h} +
  win32/Ticker.{c,h} defines the OS-specific services that
  creates/destroys a timer.
- For win32 plats, drop the long-standing use of the 'multimedia'
  API timers and implement the ticking service ourselves. Simpler
  and more flexible.
- Select.c is now solely for platforms that use select() to handle
  non-blocking I/O & thread delays. win32/AwaitEvent.c provides
  the same API on the Win32 side.
- support threadDelay on win32 platforms via worker threads.

Not yet compiled up on non-win32 platforms; will do once checked in.

Asynchronous / non-blocking I/O for Win32 platforms.

This commit introduces a Concurrent Haskell friendly view of I/O on
Win32 platforms. Through the use of a pool of worker Win32 threads, CH
threads may issue asynchronous I/O requests without blocking the
progress of other CH threads. The issuing CH thread is blocked until
the request has been serviced though.

GHC.Conc exports the primops that take care of issuing the
asynchronous I/O requests, which the IO implementation now takes
advantage of. By default, all Handles are non-blocking/asynchronous,
but should performance become an issue, having a per-Handle flag for
turning off non-blocking could easily be imagined&introduced.

[Incidentally, this thread pool-based implementation could easily be
extended to also allow Haskell code to delegate the execution of
arbitrary pieces of (potentially blocking) external code to another OS
thread. Given how relatively gnarly the locking story has turned out
to be with the 'threaded' RTS, that may not be such a bad idea.]

This commit fixes many bugs and limitations in the threaded RTS.
There are still some issues remaining, though.

The following bugs should have been fixed:

- [+] "safe" calls could cause crashes
- [+] yieldToReturningWorker/grabReturnCapability
    -     It used to deadlock.
- [+] couldn't wake blocked workers
    -     Calls into the RTS could go unanswered for a long time, and
          that includes ordinary callbacks in some circumstances.
- [+] couldn't block on an MVar and expect to be woken up by a signal
      handler
    -     Depending on the exact situation, the RTS shut down or
          blocked forever and ignored the signal.
- [+] The locking scheme in RtsAPI.c didn't work
- [+] run_thread label in wrong place (schedule())
- [+] Deadlock in GHC.Handle
    -     if a signal arrived at the wrong time, an mvar was never
          filled again
- [+] Signals delivered to the "wrong" thread were ignored or handled
      too late.

Issues:
*) If GC can move TSO objects (I don't know - can it?), then ghci
will occasionally crash when calling foreign functions, because the
parameters are stored on the TSO stack.

*) There is still a race condition lurking in the code
(both threaded and non-threaded RTS are affected):
If a signal arrives after the check for pending signals in
schedule(), but before the call to select() in awaitEvent(),
select() will be called anyway. The signal handler will be
executed much later than expected.

*) For Win32, GHC doesn't yet support non-blocking IO, so while a
thread is waiting for IO, no call-ins can happen. If the RTS is
blocked in awaitEvent, it uses a polling loop on Win32, so call-ins
should work (although the polling loop looks ugly).

*) Deadlock detection is disabled for the threaded rts, because I
don't know how to do it properly in the presence of foreign call-ins
from foreign threads.
This causes the tests conc031, conc033 and conc034 to fail.

*) "safe" is currently treated as "threadsafe". Implementing "safe" in
a way that blocks other Haskell threads is more difficult than was
thought at first. I think it could be done with a few additional lines
of code, but personally, I'm strongly in favour of abolishing the
distinction.

*) Running finalizers at program termination is inefficient - there
are two OS threads passing messages back and forth for every finalizer
that is run. Also (just as in the non-threaded case) the finalizers
are run in parallel to any remaining haskell threads and to any
foreign call-ins that might still happen.

awaitEvent: if select() reports EBADF,
always unblock all waiting threads, even if
the prior invocation of select() also
elicited an EBADF.

The 'smart' that was there previously runs
the risk of working against us (if the EBADFs are
coming from different fds), so the above
scheme is preferable (and simpler.)

Remove most #includes of system headers from Stg.h, and instead
#include any required headers directly in each RTS source file.

The idea is to (a) reduce namespace pollution from system headers that
we don't need, (c) be clearer about dependencies on system things in
the RTS, and (c) improve via-C compilation times (maybe).

In practice though, HsBase.h #includes everything anyway, so the
difference from the point of view of .hc source is minimal.  However,
this makes it easier to move to zero-includes if we wanted to (see
discussion on the FFI list; I'm still not sure that's possible but
at least this is a step in the right direction).

awaitEvent: better handling of EBADFs, i.e., don't
unconditionally barf() and exit if select() reports
an EBADF. See source code comments for details, but
in short, we attempt to unblock all threads to handle
the error condition before bailing out.

If only select() would indicate which file descriptor
that was the bad one.

(There's no good reason why select() errors other
than EBADF could also be handled this way, but let's
focus on it for now..)

Remove unused ticks_since_timestamp.

Fix a problem when a Haskell process is suspended/resumed using shell
job control in Unix.  The shell tends to put stdin back into blocking
mode before resuming the process, so we have to catch SIGCONT and put
it back into O_NONBLOCK.

Also:

  - fix a bug in the scheduler: reverse the order of the check
    for pending signals and the call to awaitEvent to block on I/O.

  - do a style sweep in Signals.c

Change the story about POSIX headers in C compilation.

Until now, all C code in the RTS and library cbits has by default been
compiled with settings for POSIXness enabled, that is:
   #define _POSIX_SOURCE   1
   #define _POSIX_C_SOURCE 199309L
   #define _ISOC9X_SOURCE
If you wanted to negate this, you'd have to define NON_POSIX_SOURCE
before including headers.

This scheme has some bad effects:

* It means that ccall-unfoldings exported via interfaces from a
  module compiled with -DNON_POSIX_SOURCE may not compile when
  imported into a module which does not -DNON_POSIX_SOURCE.

* It overlaps with the feature tests we do with autoconf.

* It seems to have caused borkage in the Solaris builds for some
  considerable period of time.

The New Way is:

* The default changes to not-being-in-Posix mode.

* If you want to force a C file into Posix mode, #include as
  the **first** include the new file ghc/includes/PosixSource.h.
  Most of the RTS C sources have this include now.

* NON_POSIX_SOURCE is almost totally expunged.  Unfortunately
  we have to retain some vestiges of it in ghc/compiler so that
  modules compiled via C on Solaris using older compilers don't
  break.

Correctly #include <windows.h>.

mingwin "fixes": getourtimeofday now returns the right units, and awaitEvent
fudged to use WinSock select(), so that although it doesn't work for file
handles, at least it doesn't cause nasty crashes; instead it just blocks.

Change the way threadDelay# is implemented.

We now use a list of sleeping threads sorted in increasing order by
the time at which they will wake up.  This avoids us having to
traverse the entire queue on each context switch.

Fix a problem where ^C wasn't recognised while waiting for I/O.

Stopped getourtimeofday() being called under mingwin.
MBlock.c: changed one instance of 128 * 1024 * 1024 into SIZE_RESERVED_POOL
and added a TODO comment (bug in is_heap_alloced(), which won't work if
more than one megablock is ever allocated).

Don't wake up threads if the select() was interrupted by a signal.

Adding an alternative to the "delay" system used for
threads that are waiting for time to pass.

This works on a target time basis, eliminating the
need to use the ticky style counter.

It is only enabled under:

#if defined(INTERPRETER) && !defined(HAVE_SETITIMER)

put a lock in the right place.

Put a giant loop around awaitEvent, to protect against
awaitEvent(rtsTrue) returning with no threads to run.

This can happen if we try to delay for some time X, and select(2)
decides to wait for a shorter time X-\delta instead.  It appears that
Solaris is more prone to doing this than Linux.

This fixes the "schedule: invalid whatNext field" crashes that people
may have seen.

- remove AllBlocked scheduler return code.  Nobody owned up to having
  created it or even knowing what it was there for.

- clean up fatal error condition handling somewhat.  The process
  exit code from a GHC program now indicates the kind of failure
  for certain kinds of exit:

	general internal RTS error	254
	program deadlocked		253
	program interrupted (ctrl-C)	252
	heap overflow			251
	main thread killed		250

  and we leave exit codes 1-199 for the user (as is traditional at MS,
  200-249 are reserved for future expansion, and may contain
  undocumented extensions :-)

Add 'par' and sparking support to the SMP implementation.

Fix bug in threadDelay, where the delay ticks could end up negative.

- don't hold the scheduler lock while doing the select, since
  we might sit in there for a long time.

- don't need the gettimeofday() hack on Linux, because select
  already returns the unslept time.

Service signal handlers if we get an EINTR from select(2).

Win32 version of awaitEvent(). Easy :)

Add select code for thread{WaitRead,WaitWrite,Delay}.