Commit 2b0918c9 authored by Simon Marlow's avatar Simon Marlow

Save a word in the info table on x86_64

Summary:
An info table with an SRT normally looks like this:

    StgWord64 srt_offset
    StgClosureInfo layout
    StgWord32 layout
    StgWord32 has_srt

But we only need 32 bits for srt_offset on x86_64, because the small
memory model requires that code segments are at most 2GB. So we can
optimise this to

    StgClosureInfo layout
    StgWord32 layout
    StgWord32 srt_offset

saving a word.  We can tell whether the info table has an SRT or not,
because zero is not a valid srt_offset, so zero still indicates that
there's no SRT.

Test Plan:
* validate
* For results, see D4632.

Reviewers: bgamari, niteria, osa1, erikd

Subscribers: thomie, carter

Differential Revision: https://phabricator.haskell.org/D4634
parent fbd28e2c
......@@ -102,8 +102,6 @@ In each case, the info table points to the SRT.
- info->srt is zero if there's no SRT, otherwise:
- info->srt == 1 and info->f.srt_offset points to the SRT
(but see TODO below, we can improve this)
e.g. for a FUN with an SRT:
StgFunInfoTable +------+
......@@ -115,6 +113,23 @@ StgStdInfoTable +------+
info->type | ... |
|------|
On x86_64, we optimise the info table representation further. The
offset to the SRT can be stored in 32 bits (all code lives within a
2GB region in x86_64's small memory model), so we can save a word in
the info table by storing the srt_offset in the srt field, which is
half a word.
On x86_64 with TABLES_NEXT_TO_CODE:
- info->srt is zero if there's no SRT, otherwise:
- info->srt is an offset from the info pointer to the SRT object
StgStdInfoTable +------+
info->layout.ptrs | |
info->layout.nptrs | |
info->srt | ------------> offset to SRT object
|------|
EXAMPLE
^^^^^^^
......@@ -281,9 +296,6 @@ implemented.
As an alternative to [FUN]: we could merge the FUN's SRT with the FUN
object itself.
TODO: make info->srt be an offset to the SRT, or zero if none (save
one word per info table that has an SRT)
Note that there are many other optimisations that we could do, but
aren't implemented. In general, we could omit any reference from an
SRT if everything reachable from it is also reachable from the other
......@@ -297,7 +309,6 @@ B = {Y,Z}
C = {X,B}
Here we could use C = {A} and therefore [Shortcut] C = A.
-}
-- ---------------------------------------------------------------------
......
......@@ -189,9 +189,13 @@ data CmmLit
-- It is also used inside the NCG during when generating
-- position-independent code.
| CmmLabelDiffOff CLabel CLabel Int Width -- label1 - label2 + offset
-- The supported Widths depend on the architecture. wordWidth
-- is supported on all architectures. Additionally W32 is
-- supported on x86_64 when using the small memory model.
-- In an expression, the width just has the effect of MO_SS_Conv
-- from wordWidth to the desired width.
--
-- In a static literal, the supported Widths depend on the
-- architecture: wordWidth is supported on all
-- architectures. Additionally W32 is supported on x86_64 when
-- using the small memory model.
| CmmBlock {-# UNPACK #-} !BlockId -- Code label
-- Invariant: must be a continuation BlockId
......@@ -224,7 +228,7 @@ cmmLitType cflags (CmmVec (l:ls)) = let ty = cmmLitType cflags l
else panic "cmmLitType: CmmVec"
cmmLitType dflags (CmmLabel lbl) = cmmLabelType dflags lbl
cmmLitType dflags (CmmLabelOff lbl _) = cmmLabelType dflags lbl
cmmLitType dflags (CmmLabelDiffOff _ _ _ width) = cmmBits width
cmmLitType _ (CmmLabelDiffOff _ _ _ width) = cmmBits width
cmmLitType dflags (CmmBlock _) = bWord dflags
cmmLitType dflags (CmmHighStackMark) = bWord dflags
......
......@@ -45,6 +45,7 @@ import Stream (Stream)
import qualified Stream
import Hoopl.Collections
import Platform
import Maybes
import DynFlags
import Panic
......@@ -188,7 +189,7 @@ mkInfoTableContents dflags
| StackRep frame <- smrep
= do { (prof_lits, prof_data) <- mkProfLits dflags prof
; let (srt_label, srt_bitmap) = mkSRTLit dflags srt
; let (srt_label, srt_bitmap) = mkSRTLit dflags info_lbl srt
; (liveness_lit, liveness_data) <- mkLivenessBits dflags frame
; let
std_info = mkStdInfoTable dflags prof_lits rts_tag srt_bitmap liveness_lit
......@@ -201,7 +202,7 @@ mkInfoTableContents dflags
| HeapRep _ ptrs nonptrs closure_type <- smrep
= do { let layout = packIntsCLit dflags ptrs nonptrs
; (prof_lits, prof_data) <- mkProfLits dflags prof
; let (srt_label, srt_bitmap) = mkSRTLit dflags srt
; let (srt_label, srt_bitmap) = mkSRTLit dflags info_lbl srt
; (mb_srt_field, mb_layout, extra_bits, ct_data)
<- mk_pieces closure_type srt_label
; let std_info = mkStdInfoTable dflags prof_lits
......@@ -211,20 +212,22 @@ mkInfoTableContents dflags
; return (prof_data ++ ct_data, (std_info, extra_bits)) }
where
mk_pieces :: ClosureTypeInfo -> [CmmLit]
-> UniqSM ( Maybe StgHalfWord -- Override the SRT field with this
, Maybe CmmLit -- Override the layout field with this
-> UniqSM ( Maybe CmmLit -- Override the SRT field with this
, Maybe CmmLit -- Override the layout field with this
, [CmmLit] -- "Extra bits" for info table
, [RawCmmDecl]) -- Auxiliary data decls
mk_pieces (Constr con_tag con_descr) _no_srt -- A data constructor
= do { (descr_lit, decl) <- newStringLit con_descr
; return ( Just (toStgHalfWord dflags (fromIntegral con_tag))
; return ( Just (CmmInt (fromIntegral con_tag)
(halfWordWidth dflags))
, Nothing, [descr_lit], [decl]) }
mk_pieces Thunk srt_label
= return (Nothing, Nothing, srt_label, [])
mk_pieces (ThunkSelector offset) _no_srt
= return (Just (toStgHalfWord dflags 0), Just (mkWordCLit dflags (fromIntegral offset)), [], [])
= return (Just (CmmInt 0 (halfWordWidth dflags)),
Just (mkWordCLit dflags (fromIntegral offset)), [], [])
-- Layout known (one free var); we use the layout field for offset
mk_pieces (Fun arity (ArgSpec fun_type)) srt_label
......@@ -235,8 +238,9 @@ mkInfoTableContents dflags
= do { (liveness_lit, liveness_data) <- mkLivenessBits dflags arg_bits
; let fun_type | null liveness_data = aRG_GEN
| otherwise = aRG_GEN_BIG
extra_bits = [ packIntsCLit dflags fun_type arity
, srt_lit, liveness_lit, slow_entry ]
extra_bits = [ packIntsCLit dflags fun_type arity ]
++ (if inlineSRT dflags then [] else [ srt_lit ])
++ [ liveness_lit, slow_entry ]
; return (Nothing, Nothing, extra_bits, liveness_data) }
where
slow_entry = CmmLabel (toSlowEntryLbl info_lbl)
......@@ -255,11 +259,21 @@ packIntsCLit dflags a b = packHalfWordsCLit dflags
mkSRTLit :: DynFlags
-> CLabel
-> Maybe CLabel
-> ([CmmLit], -- srt_label, if any
StgHalfWord) -- srt_bitmap
mkSRTLit dflags Nothing = ([], toStgHalfWord dflags 0)
mkSRTLit dflags (Just lbl) = ([CmmLabel lbl], toStgHalfWord dflags 1)
CmmLit) -- srt_bitmap
mkSRTLit dflags info_lbl (Just lbl)
| inlineSRT dflags
= ([], CmmLabelDiffOff lbl info_lbl 0 (halfWordWidth dflags))
mkSRTLit dflags _ Nothing = ([], CmmInt 0 (halfWordWidth dflags))
mkSRTLit dflags _ (Just lbl) = ([CmmLabel lbl], CmmInt 1 (halfWordWidth dflags))
-- | is the SRT offset field inline in the info table on this platform?
inlineSRT :: DynFlags -> Bool
inlineSRT dflags = platformArch (targetPlatform dflags) == ArchX86_64
&& tablesNextToCode dflags
-------------------------------------------------------------------------
--
......@@ -367,23 +381,23 @@ mkStdInfoTable
:: DynFlags
-> (CmmLit,CmmLit) -- Closure type descr and closure descr (profiling)
-> Int -- Closure RTS tag
-> StgHalfWord -- SRT length
-> CmmLit -- SRT length
-> CmmLit -- layout field
-> [CmmLit]
mkStdInfoTable dflags (type_descr, closure_descr) cl_type srt_len layout_lit
mkStdInfoTable dflags (type_descr, closure_descr) cl_type srt layout_lit
= -- Parallel revertible-black hole field
prof_info
-- Ticky info (none at present)
-- Debug info (none at present)
++ [layout_lit, type_lit]
++ [layout_lit, tag, srt]
where
prof_info
| gopt Opt_SccProfilingOn dflags = [type_descr, closure_descr]
| otherwise = []
type_lit = packHalfWordsCLit dflags (toStgHalfWord dflags (fromIntegral cl_type)) srt_len
tag = CmmInt (fromIntegral cl_type) (halfWordWidth dflags)
-------------------------------------------------------------------------
--
......
......@@ -109,7 +109,7 @@ INLINE_HEADER const StgConInfoTable *get_con_itbl(const StgClosure *c)
INLINE_HEADER StgHalfWord GET_TAG(const StgClosure *con)
{
return get_itbl(con)->has_srt;
return get_itbl(con)->srt;
}
/* -----------------------------------------------------------------------------
......
......@@ -153,6 +153,21 @@ typedef union {
} StgClosureInfo;
#if defined(x86_64_TARGET_ARCH) && defined(TABLES_NEXT_TO_CODE)
// On x86_64 we can fit a pointer offset in half a word, so put the SRT offset
// in the info->srt field directly.
#define USE_INLINE_SRT_FIELD
#endif
#if defined(USE_INLINE_SRT_FIELD)
// offset to the SRT / closure, or zero if there's no SRT
typedef StgHalfInt StgSRTField;
#else
// non-zero if there is an SRT, the offset is in the optional srt field.
typedef StgHalfWord StgSRTField;
#endif
/*
* The "standard" part of an info table. Every info table has this bit.
*/
......@@ -169,11 +184,14 @@ typedef struct StgInfoTable_ {
StgClosureInfo layout; /* closure layout info (one word) */
StgHalfWord type; /* closure type */
StgHalfWord has_srt;
StgSRTField srt;
/* In a CONSTR:
- the constructor tag
In a FUN/THUNK
- non-zero if there is an SRT
- if USE_INLINE_SRT_FIELD
- offset to the SRT (or zero if no SRT)
- otherwise
- non-zero if there is an SRT, offset is in srt_offset
*/
#if defined(TABLES_NEXT_TO_CODE)
......@@ -214,7 +232,9 @@ typedef struct StgFunInfoExtraRev_ {
StgWord bitmap;
OFFSET_FIELD(bitmap_offset); /* arg ptr/nonptr bitmap */
} b;
#if !defined(USE_INLINE_SRT_FIELD)
OFFSET_FIELD(srt_offset); /* pointer to the SRT closure */
#endif
StgHalfWord fun_type; /* function type */
StgHalfWord arity; /* function arity */
} StgFunInfoExtraRev;
......@@ -253,7 +273,9 @@ extern const StgWord stg_arg_bitmaps[];
typedef struct {
#if defined(TABLES_NEXT_TO_CODE)
#if !defined(USE_INLINE_SRT_FIELD)
OFFSET_FIELD(srt_offset); /* offset to the SRT closure */
#endif
StgInfoTable i;
#else
StgInfoTable i;
......@@ -271,16 +293,14 @@ typedef struct {
*/
typedef struct StgThunkInfoTable_ {
#if !defined(TABLES_NEXT_TO_CODE)
StgInfoTable i;
#endif
#if defined(TABLES_NEXT_TO_CODE)
#if !defined(USE_INLINE_SRT_FIELD)
OFFSET_FIELD(srt_offset); /* offset to the SRT closure */
#else
StgClosure *srt; /* pointer to the SRT closure */
#endif
#if defined(TABLES_NEXT_TO_CODE)
StgInfoTable i;
#else
StgInfoTable i;
StgClosure *srt; /* pointer to the SRT closure */
#endif
} StgThunkInfoTable;
......@@ -315,9 +335,14 @@ typedef struct StgConInfoTable_ {
* info must be a Stg[Ret|Thunk]InfoTable* (an info table that has a SRT)
*/
#if defined(TABLES_NEXT_TO_CODE)
#if x86_64_TARGET_ARCH
#define GET_SRT(info) \
((StgClosure*) (((StgWord) ((info)+1)) + (info)->srt_offset))
((StgClosure*) (((StgWord) ((info)+1)) + (info)->i.srt))
#else
#define GET_SRT(info) \
((StgClosure*) (((StgWord) ((info)+1)) + (info)->srt_offset))
#endif
#else // !TABLES_NEXT_TO_CODE
#define GET_SRT(info) ((info)->srt)
#endif
......@@ -337,8 +362,13 @@ typedef struct StgConInfoTable_ {
* info must be a StgFunInfoTable*
*/
#if defined(TABLES_NEXT_TO_CODE)
#if x86_64_TARGET_ARCH
#define GET_FUN_SRT(info) \
((StgClosure*) (((StgWord) ((info)+1)) + (info)->i.srt))
#else
#define GET_FUN_SRT(info) \
((StgClosure*) (((StgWord) ((info)+1)) + (info)->f.srt_offset))
#endif
#else
#define GET_FUN_SRT(info) ((info)->f.srt)
#endif
......
......@@ -58,7 +58,7 @@ peekItbl a0 = do
nptrs' <- (#peek StgInfoTable, layout.payload.nptrs) a0
tipe' <- (#peek StgInfoTable, type) a0
#if __GLASGOW_HASKELL__ > 804
srtlen' <- (#peek StgInfoTable, has_srt) a0
srtlen' <- (#peek StgInfoTable, srt) a0
#else
srtlen' <- (#peek StgInfoTable, srt_bitmap) a0
#endif
......@@ -398,7 +398,7 @@ pokeItbl a0 itbl = do
(#poke StgInfoTable, layout.payload.nptrs) a0 (nptrs itbl)
(#poke StgInfoTable, type) a0 (tipe itbl)
#if __GLASGOW_HASKELL__ > 804
(#poke StgInfoTable, has_srt) a0 (srtlen itbl)
(#poke StgInfoTable, srt) a0 (srtlen itbl)
#else
(#poke StgInfoTable, srt_bitmap) a0 (srtlen itbl)
#endif
......
......@@ -367,7 +367,7 @@ rts_getBool (HaskellObj p)
const StgInfoTable *info;
info = get_itbl((const StgClosure *)UNTAG_CONST_CLOSURE(p));
if (info->has_srt == 0) { // has_srt is the constructor tag
if (info->srt == 0) { // srt is the constructor tag
return 0;
} else {
return 1;
......
......@@ -536,13 +536,13 @@ loop:
switch (info->type) {
case THUNK_STATIC:
if (info->has_srt != 0) {
if (info->srt != 0) {
evacuate_static_object(THUNK_STATIC_LINK((StgClosure *)q), q);
}
return;
case FUN_STATIC:
if (info->has_srt != 0) {
if (info->srt != 0) {
evacuate_static_object(FUN_STATIC_LINK((StgClosure *)q), q);
}
return;
......
......@@ -337,7 +337,7 @@ scavenge_thunk_srt(const StgInfoTable *info)
if (!major_gc) return;
thunk_info = itbl_to_thunk_itbl(info);
if (thunk_info->i.has_srt) {
if (thunk_info->i.srt) {
StgClosure *srt = (StgClosure*)GET_SRT(thunk_info);
evacuate(&srt);
}
......@@ -351,7 +351,7 @@ scavenge_fun_srt(const StgInfoTable *info)
if (!major_gc) return;
fun_info = itbl_to_fun_itbl(info);
if (fun_info->i.has_srt) {
if (fun_info->i.srt) {
StgClosure *srt = (StgClosure*)GET_FUN_SRT(fun_info);
evacuate(&srt);
}
......@@ -1888,7 +1888,7 @@ scavenge_stack(StgPtr p, StgPtr stack_end)
p = scavenge_small_bitmap(p, size, bitmap);
follow_srt:
if (major_gc && info->i.has_srt) {
if (major_gc && info->i.srt) {
StgClosure *srt = (StgClosure*)GET_SRT(info);
evacuate(&srt);
}
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment