Use tso->id instead of tso as index to thread lables (tsos get relocated).

Please merge

Tweaks to have RTS (C) sources compile with MSVC. Apart from wibbles
related to the handling of 'inline', changed Schedule.h:POP_RUN_QUEUE()
not to use expression-level statement blocks.

Unbreak the 2nd stage of non-threaded-RTS builds by #ifdefing out a
call to wakeBlockedWorkerThread. This should probably fixed more
cleanly by taking an OO view, i.e. always defining this function, but
doing nothing in the non-threaded case. The final decision on this
issue is left to the Masters of the Threads (tm)...

Un-break non-threaded RTS
(hopefully; I have no time to test it right now)

New implementation & changed type signature of forkProcess

forkProcess now has the following type:
forkProcess :: IO () -> IO ProcessID

forkProcessAll has been removed as it is unimplementable in the threaded RTS.

forkProcess using the old type (IO (Maybe ProcessID)) was impossible to
implement correctly in the non-threaded RTS and very hard to implement
in the threaded RTS.
The new type signature allows a clean and simple implementation.

Threaded RTS:
Don't start new worker threads earlier than necessary.
After this commit, a Haskell program that uses neither forkOS nor forkIO is
really single-threaded (rather than using two OS threads internally).

Some details:
Worker threads are now only created when a capability is released, and
only when
(there are no worker threads)
	&& (there are runnable Haskell threads ||
	    there are Haskell threads blocked on IO or threadDelay)
awaitEvent can now be called from bound thread scheduling loops
(so that we don't have to create a worker thread just to run awaitEvent)

Warning police

Bound Threads
=============

Introduce a way to use foreign libraries that rely on thread local state
from multiple threads (mainly affects the threaded RTS).

See the file threads.tex in CVS at haskell-report/ffi/threads.tex
(not entirely finished yet) for a definition of this extension. A less formal
description is also found in the documentation of Control.Concurrent.

The changes mostly affect the THREADED_RTS (./configure --enable-threaded-rts),
except for saving & restoring errno on a per-TSO basis, which is also necessary
for the non-threaded RTS (a bugfix).

Detailed list of changes
------------------------

- errno is saved in the TSO object and restored when necessary:
ghc/includes/TSO.h, ghc/rts/Interpreter.c, ghc/rts/Schedule.c

- rts_mainLazyIO is no longer needed, main is no special case anymore
ghc/includes/RtsAPI.h, ghc/rts/RtsAPI.c, ghc/rts/Main.c, ghc/rts/Weak.c

- passCapability: a new function that releases the capability and "passes"
  it to a specific OS thread:
ghc/rts/Capability.h ghc/rts/Capability.c

- waitThread(), scheduleWaitThread() and schedule() get an optional
  Capability *initialCapability passed as an argument:
ghc/includes/SchedAPI.h, ghc/rts/Schedule.c, ghc/rts/RtsAPI.c

- Bound Thread scheduling (that's what this is all about):
ghc/rts/Schedule.h, ghc/rts/Schedule.c

- new Primop isCurrentThreadBound#:
ghc/compiler/prelude/primops.txt.pp, ghc/includes/PrimOps.h, ghc/rts/PrimOps.hc,
ghc/rts/Schedule.h, ghc/rts/Schedule.c

- a simple function, rtsSupportsBoundThreads, that returns true if THREADED_RTS
  is defined:
ghc/rts/Schedule.h, ghc/rts/Schedule.c

- a new implementation of forkProcess (the old implementation stays in place
  for the non-threaded case). Partially broken; works for the standard
  fork-and-exec case, but not for much else. A proper forkProcess is
  really next to impossible to implement:
ghc/rts/Schedule.c

- Library support for bound threads:
    Control.Concurrent.
      rtsSupportsBoundThreads, isCurrentThreadBound, forkOS,
      runInBoundThread, runInUnboundThread
libraries/base/Control/Concurrent.hs, libraries/base/Makefile,
libraries/base/include/HsBase.h, libraries/base/cbits/forkOS.c (new file)

Fix a bug in forkProcess: the current thread should not be on the run
queue while it is running.

Merged from STABLE (1.169.2.1).

unblockThread: barf() string wibble

New primop (mingw only),

  asyncDoProc# :: Addr# -> Addr# -> State# RealWorld-> (# State# RealWorld, Int#, Int# #)

which lets a Haskell thread hand off a pointer to external code (1st arg) for
asynchronous execution by the RTS worker thread pool. Second arg is data passed
in to the asynchronous routine. The routine is _not_ permitted to re-enter
the RTS as part of its execution.

Style nits only

Comments only

resumeThread: concCall arg is universally unused (but leave it in for now)

threadStackOverflow() wibble

Tidy up code that supports user/Haskell signal handlers.

Signals.h now defines RTS_USER_SIGNALS when this is supported,
which is then used elsewhere.

to ease replacement of the underlying allocator used by stgMallocBytes() and friends, provide (&use) RtsUtils.c:stgFree()

wibble wrt stgMallocBytes()

drop Main.h

Clean up code&interfaces that deals with timers and asynchrony:

- Timer.{c,h} now defines the platform-independent interface
  to the timing services needed by the RTS. Itimer.{c,h} +
  win32/Ticker.{c,h} defines the OS-specific services that
  creates/destroys a timer.
- For win32 plats, drop the long-standing use of the 'multimedia'
  API timers and implement the ticking service ourselves. Simpler
  and more flexible.
- Select.c is now solely for platforms that use select() to handle
  non-blocking I/O & thread delays. win32/AwaitEvent.c provides
  the same API on the Win32 side.
- support threadDelay on win32 platforms via worker threads.

Not yet compiled up on non-win32 platforms; will do once checked in.

This commit fixes many bugs and limitations in the threaded RTS.
There are still some issues remaining, though.

The following bugs should have been fixed:

- [+] "safe" calls could cause crashes
- [+] yieldToReturningWorker/grabReturnCapability
    -     It used to deadlock.
- [+] couldn't wake blocked workers
    -     Calls into the RTS could go unanswered for a long time, and
          that includes ordinary callbacks in some circumstances.
- [+] couldn't block on an MVar and expect to be woken up by a signal
      handler
    -     Depending on the exact situation, the RTS shut down or
          blocked forever and ignored the signal.
- [+] The locking scheme in RtsAPI.c didn't work
- [+] run_thread label in wrong place (schedule())
- [+] Deadlock in GHC.Handle
    -     if a signal arrived at the wrong time, an mvar was never
          filled again
- [+] Signals delivered to the "wrong" thread were ignored or handled
      too late.

Issues:
*) If GC can move TSO objects (I don't know - can it?), then ghci
will occasionally crash when calling foreign functions, because the
parameters are stored on the TSO stack.

*) There is still a race condition lurking in the code
(both threaded and non-threaded RTS are affected):
If a signal arrives after the check for pending signals in
schedule(), but before the call to select() in awaitEvent(),
select() will be called anyway. The signal handler will be
executed much later than expected.

*) For Win32, GHC doesn't yet support non-blocking IO, so while a
thread is waiting for IO, no call-ins can happen. If the RTS is
blocked in awaitEvent, it uses a polling loop on Win32, so call-ins
should work (although the polling loop looks ugly).

*) Deadlock detection is disabled for the threaded rts, because I
don't know how to do it properly in the presence of foreign call-ins
from foreign threads.
This causes the tests conc031, conc033 and conc034 to fail.

*) "safe" is currently treated as "threadsafe". Implementing "safe" in
a way that blocks other Haskell threads is more difficult than was
thought at first. I think it could be done with a few additional lines
of code, but personally, I'm strongly in favour of abolishing the
distinction.

*) Running finalizers at program termination is inefficient - there
are two OS threads passing messages back and forth for every finalizer
that is run. Also (just as in the non-threaded case) the finalizers
are run in parallel to any remaining haskell threads and to any
foreign call-ins that might still happen.

Shortcut when switching evaluators: instead of going round the normal
scheduler loop, just cut to the chase and run the thread using the
other evaluator.

This avoids doing stack squeezing each time we switch evaluators,
which is an O(n) operation these days, whereas it used to be O(n) the
first time, and O(1) thereafter if the stack hadn't changed too much.
This is a problem that we should perhaps address separately, but for
now the workaround should provide a speed boost to GHCi on the HEAD.

Merge the eval-apply-branch on to the HEAD
------------------------------------------

This is a change to GHC's evaluation model in order to ultimately make
GHC more portable and to reduce complexity in some areas.

At some point we'll update the commentary to describe the new state of
the RTS.  Pending that, the highlights of this change are:

  - No more Su.  The Su register is gone, update frames are one
    word smaller.

  - Slow-entry points and arg checks are gone.  Unknown function calls
    are handled by automatically-generated RTS entry points (AutoApply.hc,
    generated by the program in utils/genapply).

  - The stack layout is stricter: there are no "pending arguments" on
    the stack any more, the stack is always strictly a sequence of
    stack frames.

    This means that there's no need for LOOKS_LIKE_GHC_INFO() or
    LOOKS_LIKE_STATIC_CLOSURE() any more, and GHC doesn't need to know
    how to find the boundary between the text and data segments (BIG WIN!).

  - A couple of nasty hacks in the mangler caused by the neet to
    identify closure ptrs vs. info tables have gone away.

  - Info tables are a bit more complicated.  See InfoTables.h for the
    details.

  - As a side effect, GHCi can now deal with polymorphic seq.  Some bugs
    in GHCi which affected primitives and unboxed tuples are now
    fixed.

  - Binary sizes are reduced by about 7% on x86.  Performance is roughly
    similar, some programs get faster while some get slower.  I've seen
    GHCi perform worse on some examples, but haven't investigated
    further yet (GHCi performance *should* be about the same or better
    in theory).

  - Internally the code generator is rather better organised.  I've moved
    info-table generation from the NCG into the main codeGen where it is
    shared with the C back-end; info tables are now emitted as arrays
    of words in both back-ends.  The NCG is one step closer to being able
    to support profiling.

This has all been fairly thoroughly tested, but no doubt I've messed
up the commit in some way.

Fix a race condition/possible deadlock in the threaded rts:

If a callback into haskell finished before waitThread_() was called,
the signal was lost  ans waitThread_() waited indefinitely.

Solution: Don't release the sched_mutex between calls to scheduleThread_
and waitThread_.

Please note that the scheduler API function waitThread is still possibly
affected by this race condition. It's used in rts_mainEvalIO (I think that's
safe) and in finishAllThreads (this looks dangerous, but finishAllThreads is
never used).

change the types of cmp_thread, rts_getThreadId, and labelThread to
take StgPtr rather than StgTSO *, since the compiler now has no
distinction between these two types in the back end.

I also noticed that labelThread need not be a primitive: it could just
as well be a normal C function called by the FFI, but I haven't made
that change.

Fix a scheduling/GC bug, spotted by Wolfgang Thaller.  If a main
thread completes, and a GC runs before the return (from rts_evalIO())
happens, then the thread might be GC'd before we get a chance to
extract its return value, leading to barf("main thread has been GC'd")
from the garbage collector.

The fix is to treat all main threads which have completed as roots:
this is logically the right thing to do, because these threads must be
retained by virtue of holding the return value, and this is a property of
main threads only.

ifdefs for mingw32 to get rid of signal functions.

The GC wasn't properly marking pending signal handlers, which could
lead to "EVACUATED object entered!" errors.  Also, a race occurs if a
signal arrives during GC.  Two fixes:

  (a) mark all pending signal handlers during GC, and
  (b) block signals during GC

MERGE TO STABLE

Remove duplicate sched_belch() call from raiseAsync().

Global and common variable sweep: staticize many variables that don't
need to be globally visible.

Expose printAllThreads() in non-DEBUG builds;
of some use when debugging ConcHask code and
want a dump of the current state of the system's
threads.

Not provided via a library, so to use, do

 foreign import call unsafe printAllThreads :: IO ()

forkProcess():
 be less ambitious, and don't kill any main_threads.
 Killing them, and possibly replacing them on the
 main_threads with the current one, turned out to
 run into problems of its own (see comments.)

 The hope/wish is that the current behaviour of
 forkProcess will suffice.

forkProcess():
 - fix bug which left run_queue_tl in a bad state.
 - be better behaved wrt 'main threads', i.e.,
   if the killing thread isn't the main thread,
   make sure it's hooked up to main_threads +
   correctly signal the completion/killing of
   any main threads.

Remove most #includes of system headers from Stg.h, and instead
#include any required headers directly in each RTS source file.

The idea is to (a) reduce namespace pollution from system headers that
we don't need, (c) be clearer about dependencies on system things in
the RTS, and (c) improve via-C compilation times (maybe).

In practice though, HsBase.h #includes everything anyway, so the
difference from the point of view of .hc source is minimal.  However,
this makes it easier to move to zero-includes if we wanted to (see
discussion on the FFI list; I'm still not sure that's possible but
at least this is a step in the right direction).

Fix a GC bug.  In a "large block", only the block descriptor for the
head of the block has the fields step, gen_no and flags set.  So
whenever we want one of these fields in the descriptor for a random
object anywhere in the large block, we have to check whether it is in
the head block, and if not follow the link to the head block
descriptor.

evacuate() was doing this correctly, but isAlive() wasn't (and
goodness knows what other places are broken in this way - I identified
several other possible cases of the same bug).

So to try to make things more robust, when we allocate a large block
we now initialise the step, gen_no, and flags fields in the descriptor
for *every* sub-block, not just the first one.  Now, as long as you
only want one of these fields from the descriptor, there's no need to
try to find the block head.  evacuate() gets minutely faster, and
hopefully multiple obscure bugs are fixed by this.

- Make TSO "stable" again: The thread label was changing the size of the
   TSO if you were building a debugging-RTS, leading to binary
   incompatibility. Now we map TSOs to strings using Hash.c.

- API change for labelThread: Label arbitrary threads.

When handling external call-ins (via the RTS API) in
the multi-threaded case, add the StgMainThread that
the external thread is going to block waiting on
to the main_threads list prior to scheduling the new
worker thread.

Do this by having the scheduler provide a new entry
point, scheduleWaitThread().

Fixes a bug/race condition spotted by Wolfgang Thaller
(see scheduleWaitThread() comment) + enables a little
tidier interface between RtsAPI and Schedule.

Add missing va_end's to va_start's.

Have createThread() use a separate mutex to ensure unique
allocation of ThreadIds. A less-than-light solution, but
cleaner than trying to reuse sched_mutex for this purpose.

As a result of calling exitScheduler(), enter into a
'shutting-down' state, so that the subsequent (sequential)
execution of finalisers won't get stuck inside the
Scheduler waiting for more work.

Cleanest way I could think of solving this problem on a
Friday afternoon.