Skip to content

Use `+RTS -G1` for more stable residency measurements

Tests that measure peak_megabytes_allocated and max_bytes_used can be very sensitive to small changes in, well, just about anything: order of command line flags, what should be irrelevant compiler patches, minor changes in the test file to compile.

From Note [residency] in testsuite/tests/perf/compiler/all.T:

# Residency (peak_megabytes_allocated and max_bytes_used) is sensitive
# to when the major GC runs, which makes it inherently inaccurate.
# Sometime an innocuous change somewhere can shift things around such
# that the samples occur at a different time, and the residency
# appears to change (up or down) when the underlying profile hasn't
# really changed.

I propose setting +RTS -G1 for these tests. This seems to make them more robust to small changes, as can be seen in the following two graphs, adapted from the test for #9675:

https://ghc.haskell.org/trac/ghc/attachment/ticket/9675/T9675-peak_megabytes_allocated.png https://ghc.haskell.org/trac/ghc/attachment/ticket/9675/T9675-max_bytes_used.png

This shows the peak_megabytes_allocated and max_bytes_used as a function of the number fields when compiling a single record:

data Foo = Foo
   { field1 :: Int -> Int
   , field2 :: Int -> Int
   ...

The default GHC -O line is wiggling up and down, while the GHC -O +RTS -G1 line is nice and smooth, which is what we want.

I don't know if making this change defeats the purpose of some tests, which is why I'm opening this ticket.

Trac metadata
Trac field Value
Version 7.10.1
Type Task
TypeOfFailure OtherFailure
Priority normal
Resolution Unresolved
Component Test Suite
Test case
Differential revisions
BlockedBy
Related
Blocking
CC ezyang, thoughtpolice
Operating system
Architecture
To upload designs, you'll need to enable LFS and have an admin enable hashed storage. More information