HSOC Eventlog live profiling

Hi, it is ticket for results of my Haskell Summer of Code 2016 project [1].

The goal is provide feature for redirection of events to external profilers without an application termination.

Proposed changes:

• Changing eventlog file descriptor in runtime.

/*
 * Set custom file stream for global event log sink.
 *
 * The function overwrites previous event log file pointer. Previouss
 * sink is closed only if closePrev flag is on.
 *
 * Writing to the sink is protected by global mutex.
 *
 * The function puts header to the new sink only when emitHeader flag
 * is on. User might not want the header if it is switching to
 * already existed eventlog handle that was switched away recently.
 */
void rts_setEventLogSink(FILE *sink,
                         StgBool closePrev,
                         StgBool emitHeader);

/*
 * Get current file stream that is used for global event log sink.
 *
 * You shouldn't do anything with the pointer until
 * rts_setEventLogSink(otherFileOrNull, false) is called. After that
 * you can do anything with the file stream.
 */
FILE* rts_getEventLogSink(void);

• Special flag -lm for RTS to store eventlog in memory chunks. User space functions to retrieve the chunks from memory.

/*
 * If RTS started with '-lm' flag then eventlog is stored in memory buffer.
 *
 * The function allows to pop chunks of the buffer. Return value of 0 means
 * that there is no any filled chunk of data.
 *
 * If the function returns nonzero value the parameter contains full chunk
 * of eventlog data with size of the returned value. Caller must free the
 * buffer, the buffer isn't referenced anywhere anymore.
 *
 * If nobody calls the function with '-lm' flag then the memory is kinda
 * to be exhausted.
 */
StgWord64 rts_getEventLogChunk(StgInt8** ptr);

• Feature for dynamic resize of eventlog buffers.

/*
 * Reallocate inner buffers to match the new size. The size should be not
 * too small to contain at least one event.
 *
 * If RTS started with '-lm' the chunks of memory buffer is also resized.
 */
void rts_resizeEventLog(StgWord64 size);

/*
 * Return current size of eventlog buffers.
 */
StgWord64 rts_getEventLogBuffersSize(void);

• Bindings to new features in Debug.Trace in base package.

-- | The 'setEventLogCFile' function changes current sink of the eventlog, if eventlog
-- profiling is available and enabled at runtime.
--
-- The second parameter defines whether old sink should be finalized and closed or not. 
-- Preserving it could be helpful for temporal redirection of eventlog data into not 
-- standard sink and then restoring to the default file sink.
--
-- The third parameter defines whether new header section should be emitted to the new
-- sink. Emitting header to already started eventlog streams will corrupt the structure 
-- of eventlog format.
--
-- The function is more low-level than 'setEventLogHandle' but doesn't recreate underlying
-- file descriptor and is intended to use with 'getEventLogCFile' to save and restore 
-- current sink of the eventlog.
--
-- @since 4.10.0.0
setEventLogCFile :: Ptr CFile -> Bool -> Bool -> IO ()

-- | The 'getEventLogCFile' function returns current sink of the eventlog, if eventlog
-- profiling is available and enabled at runtime.
--
-- The function is intented to be used with 'setEventLogCFile' to save and restore 
-- current sink of the eventlog.
--
-- @since 4.10.0.0
getEventLogCFile :: IO (Ptr CFile)

-- | Setting size of internal eventlog buffers. The size should be not
-- too small to contain at least one event.
--
-- If RTS started with '-lm' the chunks of memory buffer is also resized.
--
-- The larger the buffers the lesser overhead from event logging, but 
-- larger delays between data dumps.
--
-- See also: 'getEventLogChunk', 'getEventLogBufferSize'
setEventLogBufferSize :: Word -> IO ()

-- | Getting size of internal eventlog buffers.
--
-- See also: 'setEventLogBufferSize', 'getEventLogChunk'
getEventLogBufferSize :: IO Word

-- | Get next portion of the eventlog data.
--
-- If RTS started with '-lm' flag then eventlog is stored in memory buffer.
-- 
-- The function allows to pop chunks out of the buffer. Return value of Nothing 
-- means that there is no any filled chunk of data.
--
-- If the function returns nonzero value the parameter contains full chunk 
-- of eventlog data with size of the returned value. Caller must free the
-- buffer with 'free' from 'Foreign.Marshal.Alloc', the buffer isn't referenced 
-- anywhere anymore.
--
-- If nobody calls the function with '-lm' flag on then the memory is kinda
-- to be exhausted.
--
-- If '-lm' flag is off, the function returns always 'Nothing'.
--
-- See also: 'setEventLogBufferSize'
getEventLogChunk :: IO (Maybe CStringLen)

References:

1. HSOC project description
2. Design choices for implementation

Trac metadata

Trac field	Value
Version	8.0.1
Type	FeatureRequest
TypeOfFailure	OtherFailure
Priority	normal
Resolution	Unresolved
Component	Runtime System
Test case
Differential revisions
BlockedBy
Related
Blocking
CC	simonmar
Operating system
Architecture