GitLab

setupRtsFlags(): don't overwrite argv[0] with its basename:
   - argv[] may not point to writeable memory
   - System.Environment.getProgName strips off the 'dirname' portion
     anyway.
   - Not possible to get at the untransformed argv[0] from
     Haskell code, should such a need arise.

Uses of prog_argv[0] within the RTS has now been replaced with prog_name,
which is the basename of prog_argv[0].

setlocale(LC_ALL,""), to enable the locale support in libc.

Tidy up code that supports user/Haskell signal handlers.

Signals.h now defines RTS_USER_SIGNALS when this is supported,
which is then used elsewhere.

Clean up code&interfaces that deals with timers and asynchrony:

- Timer.{c,h} now defines the platform-independent interface
  to the timing services needed by the RTS. Itimer.{c,h} +
  win32/Ticker.{c,h} defines the OS-specific services that
  creates/destroys a timer.
- For win32 plats, drop the long-standing use of the 'multimedia'
  API timers and implement the ticking service ourselves. Simpler
  and more flexible.
- Select.c is now solely for platforms that use select() to handle
  non-blocking I/O & thread delays. win32/AwaitEvent.c provides
  the same API on the Win32 side.
- support threadDelay on win32 platforms via worker threads.

Not yet compiled up on non-win32 platforms; will do once checked in.

Asynchronous / non-blocking I/O for Win32 platforms.

This commit introduces a Concurrent Haskell friendly view of I/O on
Win32 platforms. Through the use of a pool of worker Win32 threads, CH
threads may issue asynchronous I/O requests without blocking the
progress of other CH threads. The issuing CH thread is blocked until
the request has been serviced though.

GHC.Conc exports the primops that take care of issuing the
asynchronous I/O requests, which the IO implementation now takes
advantage of. By default, all Handles are non-blocking/asynchronous,
but should performance become an issue, having a per-Handle flag for
turning off non-blocking could easily be imagined&introduced.

[Incidentally, this thread pool-based implementation could easily be
extended to also allow Haskell code to delegate the execution of
arbitrary pieces of (potentially blocking) external code to another OS
thread. Given how relatively gnarly the locking story has turned out
to be with the 'threaded' RTS, that may not be such a bad idea.]

Quick fix for profiling after I broke it in rev. 1.68.

The profiling system needs some initialisation after module
registration (hs_add_root()).  Since module registration now happens
later, the profiling system was getting initialised too early before
any cost centers had been registered.  This is a quick fix; we still
can't handle multiple hs_add_root()s with profiling.

- re-instate setProgArgv, it is used in System.Environment (bah, could
  have sworn I grepped for it and found nothing...)

- Remove init_stack symbol from the Linker's symbol table; this is
  now static.

Flesh out support for hs_init() and hs_exit() according to the latest
FFI spec.

For GHC, I also added:

  hs_add_root( void (*fn)(void) );

which is used to specify the root module.  This *must* be called prior
to invoking any Haskell functions.

The previous way of doing things still works:

  startupHaskell( argc, argv, root );

but the right way to do this is now

  hs_init( &argc, &argv );
  hs_add_root( root );

It is possible to invoke hs_add_root() multiple times with different
roots.

- setProgArgv() has been removed; it was unused and looks like it was
  there to support STG Hugs.

Merge the eval-apply-branch on to the HEAD
------------------------------------------

This is a change to GHC's evaluation model in order to ultimately make
GHC more portable and to reduce complexity in some areas.

At some point we'll update the commentary to describe the new state of
the RTS.  Pending that, the highlights of this change are:

  - No more Su.  The Su register is gone, update frames are one
    word smaller.

  - Slow-entry points and arg checks are gone.  Unknown function calls
    are handled by automatically-generated RTS entry points (AutoApply.hc,
    generated by the program in utils/genapply).

  - The stack layout is stricter: there are no "pending arguments" on
    the stack any more, the stack is always strictly a sequence of
    stack frames.

    This means that there's no need for LOOKS_LIKE_GHC_INFO() or
    LOOKS_LIKE_STATIC_CLOSURE() any more, and GHC doesn't need to know
    how to find the boundary between the text and data segments (BIG WIN!).

  - A couple of nasty hacks in the mangler caused by the neet to
    identify closure ptrs vs. info tables have gone away.

  - Info tables are a bit more complicated.  See InfoTables.h for the
    details.

  - As a side effect, GHCi can now deal with polymorphic seq.  Some bugs
    in GHCi which affected primitives and unboxed tuples are now
    fixed.

  - Binary sizes are reduced by about 7% on x86.  Performance is roughly
    similar, some programs get faster while some get slower.  I've seen
    GHCi perform worse on some examples, but haven't investigated
    further yet (GHCi performance *should* be about the same or better
    in theory).

  - Internally the code generator is rather better organised.  I've moved
    info-table generation from the NCG into the main codeGen where it is
    shared with the C back-end; info tables are now emitted as arrays
    of words in both back-ends.  The NCG is one step closer to being able
    to support profiling.

This has all been fairly thoroughly tested, but no doubt I've messed
up the commit in some way.

Global and common variable sweep: staticize many variables that don't
need to be globally visible.

Remove most #includes of system headers from Stg.h, and instead
#include any required headers directly in each RTS source file.

The idea is to (a) reduce namespace pollution from system headers that
we don't need, (c) be clearer about dependencies on system things in
the RTS, and (c) improve via-C compilation times (maybe).

In practice though, HsBase.h #includes everything anyway, so the
difference from the point of view of .hc source is minimal.  However,
this makes it easier to move to zero-includes if we wanted to (see
discussion on the FFI list; I'm still not sure that's possible but
at least this is a step in the right direction).

- Make TSO "stable" again: The thread label was changing the size of the
   TSO if you were building a debugging-RTS, leading to binary
   incompatibility. Now we map TSOs to strings using Hash.c.

- API change for labelThread: Label arbitrary threads.

As a result of calling exitScheduler(), enter into a
'shutting-down' state, so that the subsequent (sequential)
execution of finalisers won't get stuck inside the
Scheduler waiting for more work.

Cleanest way I could think of solving this problem on a
Friday afternoon.

When the OS thread is all out of work, shut down in a gentler
manner by calling shutdownHaskell() rather than
shutdownHaskellAndExit(). Give up the sched_mutex lock also.

startupHaskell: Scheduler startup now handles task creation

- Add a new type of restriction: -hC, which restricts to closures
  whose CCS contains the specified CCs *anywhere* (not just at the
  top).

- Complain if the user tries to request both retainer and biographical
  profiling.  We don't support both simultaneously, because they use
  the same header word in the closure.

- Allow for the fact that the heap might contain some closures which
  don't have a valid retainer set during the heap census.  The only
  known closures of this kind so far are DEAD_WEAK closures.

- Some cruft-removal and renaming of functions to follow conventions.

startupHaskell: if invoked more than once, do run initModules() each time,
as there might be more than one 'root module'.

Rearranged how the virtual timer 'subsystem' is brought up and down:

* "int startVirtTimer(nat ms)" now starts the vtimer/ticker.
* "int stopVirtTimer()" shuts down the ticker.

i.e., removed install_vtalrm_handler() and initialize_virtual_timer().

Profiling cleanup.

This commit eliminates some duplication in the various heap profiling
subsystems, and generally centralises much of the machinery.  The key
concept is the separation of a heap *census* (which is now done in one
place only instead of three) from the calculation of retainer sets.
Previously the retainer profiling code also did a heap census on the
fly, and lag-drag-void profiling had its own census machinery.

Value-adds:

   - you can now restrict a heap profile to certain retainer sets,
     but still display by cost centre (or type, or closure or
     whatever).

   - I've added an option to restrict the maximum retainer set size
     (+RTS -R<size>, defaulting to 8).

   - I've cleaned up the heap profiling options at the request of
     Simon PJ.  See the help text for details.  The new scheme
     is backwards compatible with the old.

   - I've removed some odd bits of LDV or retainer profiling-specific
     code from various parts of the system.

   - the time taken doing heap censuses (and retainer set calculation)
     is now accurately reported by the RTS when you say +RTS -Sstderr.

Still to come:

   - restricting a profile to a particular biography
     (lag/drag/void/use).  This requires keeping old heap censuses
     around, but the infrastructure is now in place to do this.

Retainer Profiling / Lag-drag-void profiling.

This is mostly work by Sungwoo Park, who spent a summer internship at
MSR Cambridge this year implementing these two types of heap profiling
in GHC.

Relative to Sungwoo's original work, I've made some improvements to
the code:

   - it's now possible to apply constraints to retainer and LDV profiles
     in the same way as we do for other types of heap profile (eg.
     +RTS -hc{foo,bar} -hR -RTS gives you a retainer profiling considering
     only closures with cost centres 'foo' and 'bar').

   - the heap-profile timer implementation is cleaned up.

   - heap profiling no longer has to be run in a two-space heap.

   - general cleanup of the code and application of the SDM C coding
     style guidelines.

Profiling will be a little slower and require more space than before,
mainly because closures have an extra header word to support either
retainer profiling or LDV profiling (you can't do both at the same
time).

We've used the new profiling tools on GHC itself, with moderate
success.  Fixes for some space leaks in GHC to follow...

Fix the large block allocation bug (Yay!)
-----------------------------------------

In order to do this, I had to

 1. in each heap-check failure branch, return the amount of heap
    actually requested, in a known location (I added another slot
    in StgRegTable called HpAlloc for this purpose).  This is
    useful for other reasons - in particular it makes it possible
    to get accurate allocation statistics.

 2. In the scheduler, if a heap check fails and we wanted more than
    BLOCK_SIZE_W words, then allocate a special large block and place
    it in the nursery.  The nursery now has to be double-linked so
    we can insert the new block in the middle.

 3. The garbage collector has to be able to deal with multiple objects
    in a large block.  It turns out that this isn't a problem as long as
    the large blocks only occur in the nursery, because we always copy
    objects from the nursery during GC.  One small change had to be
    made: in evacuate(), we may need to follow the link field from the
    block descriptor to get to the block descriptor for the head of a
    large block.

 4. Various other parts of the storage manager had to be modified
    to cope with a nursery containing a mixture of block sizes.

Point (3) causes a slight pessimization in the garbage collector.  I
don't see a way to avoid this.  Point (1) causes some code bloat (a
rough measurement is around 5%), so to offset this I made the
following change which I'd been meaning to do for some time:

  - Store the values of some commonly-used absolute addresses
    (eg. stg_update_PAP) in the register table.  This lets us use
    shorter instruction forms for some absolute jumps and saves some
    code space.

  - The type of Capability is no longer the same as an StgRegTable.
    MainRegTable renamed to MainCapability.  See Regs.h for details.

Other minor changes:

  - remove individual declarations for the heap-check-failure jump
    points, and declare them all in StgMiscClosures.h instead.  Remove
    HeapStackCheck.h.

Updates to the native code generator to follow.

Fix a problem when a Haskell process is suspended/resumed using shell
job control in Unix.  The shell tends to put stdin back into blocking
mode before resuming the process, so we have to catch SIGCONT and put
it back into O_NONBLOCK.

Also:

  - fix a bug in the scheduler: reverse the order of the check
    for pending signals and the call to awaitEvent to block on I/O.

  - do a style sweep in Signals.c

THIS CHANGE AFFECTS ALL OBJECT FILES COMPILED FROM HASKELL.
Please say "make -C ghc/lib/std clean; make -C hslibs clean".

This commit eliminates spurious warning messages when compiling on
the Alpha.  There are two kinds of spurious warning messages:

(1) gcc: -noprefix_recognition: linker input file unused since linking not done

    This warning is because we pass the flag "-Xlinker -noprefix_recognition"
    to gcc.  We remove this warning by no longer passing the flag to gcc,
    and by removing the reason we were passing the flag in the first place:
    __init_* is now renamed to __stginit_*.

(2) .../includes/Regs.h: warning: call-clobbered register used for global
    register variable

    This warning and all other warnings except (1), we eliminate by
    passing the -w flag to gcc.

MERGE TO STABLE BRANCH

Change the story about POSIX headers in C compilation.

Until now, all C code in the RTS and library cbits has by default been
compiled with settings for POSIXness enabled, that is:
   #define _POSIX_SOURCE   1
   #define _POSIX_C_SOURCE 199309L
   #define _ISOC9X_SOURCE
If you wanted to negate this, you'd have to define NON_POSIX_SOURCE
before including headers.

This scheme has some bad effects:

* It means that ccall-unfoldings exported via interfaces from a
  module compiled with -DNON_POSIX_SOURCE may not compile when
  imported into a module which does not -DNON_POSIX_SOURCE.

* It overlaps with the feature tests we do with autoconf.

* It seems to have caused borkage in the Solaris builds for some
  considerable period of time.

The New Way is:

* The default changes to not-being-in-Posix mode.

* If you want to force a C file into Posix mode, #include as
  the **first** include the new file ghc/includes/PosixSource.h.
  Most of the RTS C sources have this include now.

* NON_POSIX_SOURCE is almost totally expunged.  Unfortunately
  we have to retain some vestiges of it in ghc/compiler so that
  modules compiled via C on Solaris using older compilers don't
  break.

wibble - silence CPP warnings

-*- outline -*-
Time-stamp: <Thu Mar 22 2001 03:50:16 Stardate: [-30]6365.79 hwloidl>

This commit covers changes in GHC to get GUM (way=mp) and GUM/GdH (way=md)
working. It is a merge of my working version of GUM, based on GHC 4.06,
with GHC 4.11. Almost all changes are in the RTS (see below).

GUM is reasonably stable, we used the 4.06 version in large-ish programs for
recent papers. Couple of things I want to change, but nothing urgent.
GUM/GdH has just been merged and needs more testing. Hope to do that in the
next weeks. It works in our working build but needs tweaking to run.
GranSim doesn't work yet (*sigh*). Most of the code should be in, but needs
more debugging.

ToDo: I still want to make the following minor modifications before the release
- Better wrapper skript for parallel execution [ghc/compiler/main]
- Update parallel docu: started on it but it's minimal [ghc/docs/users_guide]
- Clean up [nofib/parallel]: it's a real mess right now (*sigh*)
- Update visualisation tools (minor things only IIRC) [ghc/utils/parallel]
- Add a Klingon-English glossary

* RTS:

Almost all changes are restricted to ghc/rts/parallel and should not
interfere with the rest. I only comment on changes outside the parallel
dir:

- Several changes in Schedule.c (scheduling loop; createThreads etc);
  should only affect parallel code
- Added ghc/rts/hooks/ShutdownEachPEHook.c
- ghc/rts/Linker.[ch]: GUM doesn't know about Stable Names (ifdefs)!!
- StgMiscClosures.h: END_TSO_QUEUE etc now defined here (from StgMiscClosures.hc)
                     END_ECAF_LIST was missing a leading stg_
- SchedAPI.h: taskStart now defined in here; it's only a wrapper around
              scheduleThread now, but might use some init, shutdown later
- RtsAPI.h: I have nuked the def of rts_evalNothing

* Compiler:

- ghc/compiler/main/DriverState.hs
  added PVM-ish flags to the parallel way
  added new ways for parallel ticky profiling and distributed exec

- ghc/compiler/main/DriverPipeline.hs
  added a fct run_phase_MoveBinary which is called with way=mp after linking;
  it moves the bin file into a PVM dir and produces a wrapper script for
  parallel execution
  maybe cleaner to add a MoveBinary phase in DriverPhases.hs but this way
  it's less intrusive and MoveBinary makes probably only sense for mp anyway

* Nofib:

- nofib/spectral/Makefile, nofib/real/Makefile, ghc/tests/programs/Makefile:
  modified to skip some tests if HWL_NOFIB_HACK is set; only tmp to record
  which test prgs cause problems in my working build right now

Bite the bullet and make GHCi support non-optional in the RTS.  GHC
4.11 should be able to build GHCi without any additional tweaks now.

- the Linker is split into two parts: LinkerBasic.c, containing the
  routines required by the rest of the RTS, and Linker.c, containing
  the linker proper, which is not referred to from the rest of the RTS.
  Only Linker.c requires -ldl, so programs which don't make use of the
  linker (everything except GHC, in other words) won't need -ldl.

Remove all vestiges of INTERPRETER and __HUGS__.

Declare the 3rd argument to startupHaskell as a function pointer
rather than a void *, to avoid accidental mis-declarations of
__init_Foo symbols.

match changes in the Stats API

merge recent changes from before-ghci-branch onto the HEAD

Add a basic "front panel" for GHC-compiled programs.

How to use it:

	- re-autoconf & configure to detect GTK+

	- add "GhcRtsWithFrontPanel = YES" to mk/build.mk

	- rebuild the RTS

	- compile up a program, add `gtk-config --libs` to the
	  link command line

	- run with program with +RTS -f,

	- sit back & watch the show :-)  Programs with lots of
	  heap-resident data are the most interesting.  For extra
	  kicks, turn up the number of generations & steps like so:
	  +RTS -f -G5 -T3.

	- Bootstrap your compiler, and see in glorious technicolor
	  just how much of a lumbering beast GHC really is.

This is a work in progress.  There's lots more stuff we could display
on the panel: suggestions/comments are of course welcome.  The window
layout was designed with GLADE, I'll commit the config file shortly.

I haven't quite figured out how we're going to integrate this with the
release yet (ie. whether we'll distribute two separate RTS's or what).

Initialise the object linker if we're in GHCi.

Fix for initModules when unregisterised.

Pull init_stack outside #ifdef INTERPRETER.

Make getArgs and getProgName behave identically in combined and standalone
modes.

Don't use MainRegTable to communicate Sp to stg_init, use a
dynamically-allocated Capability instead (works with SMP).

misc profiling cleanups

Numerous changes in the RTS to get GUM-4.06 working (currently works with
parfib-ish programs). Most changes are isolated in the rts/parallel dir.

rts/parallel/:
  The most important changes are a rewrite of the (un-)packing code (Pack.c)
  and changes in LAGA, GALA table operations (Global.c) expecially in
  rebuilding the tables during GC.

rts/:
  Minor changes in Schedule.c, GC.c (interface to par specific root marking
  and evacuation), and lots of additions to Sanity.c (surprise ;-)
  Main.c change for startup: I use a new function rts_evalNothing to
  start non-main-PEs in a PAR || SMP setup (RtsAPI.c)

includes/:
  Updated GranSim macros in PrimOps.h.

lib/std:
  Few changes in PrelHandle.c etc replacing ForeignObj by Addr in a PAR
  setup (we still don't support ForeignObjs or WeakPtrs in GUM).
  Typically use
    #define FILE_OBJECT	    Addr
  when dealing with files.

hslibs/lang/:
  Same as above (in Foreign(Obj).lhs, Weak.lhs, IOExts.lhs etc).

-- HWL

HEADS UP!!!

change the type of startupHaskell():

   void startupHaskell  ( int argc, char *argv[], void *init_root );

the extra parameter is a pointer to the initialisation function for
the root module in the program.  eg., Main.c now passes __init_Main for
this parameter.  It can be left as NULL if there is no root module.

This interface may need to be revised, since in some circumstances
there may be more than one "root module".

Sigbjorn: H/Direct will need some changes to stay in sync here.