Use `+RTS -G1` for more stable residency measurements
Tests that measure peak_megabytes_allocated and max_bytes_used can be very sensitive to small changes in, well, just about anything: order of command line flags, what should be irrelevant compiler patches, minor changes in the test file to compile.
From Note [residency] in testsuite/tests/perf/compiler/all.T:
# Residency (peak_megabytes_allocated and max_bytes_used) is sensitive
# to when the major GC runs, which makes it inherently inaccurate.
# Sometime an innocuous change somewhere can shift things around such
# that the samples occur at a different time, and the residency
# appears to change (up or down) when the underlying profile hasn't
# really changed.I propose setting +RTS -G1 for these tests. This seems to make them more robust to small changes, as can be seen in the following two graphs, adapted from the test for #9675:
https://ghc.haskell.org/trac/ghc/attachment/ticket/9675/T9675-peak_megabytes_allocated.png https://ghc.haskell.org/trac/ghc/attachment/ticket/9675/T9675-max_bytes_used.png
This shows the peak_megabytes_allocated and max_bytes_used as a function of the number fields when compiling a single record:
data Foo = Foo
{ field1 :: Int -> Int
, field2 :: Int -> Int
...The default GHC -O line is wiggling up and down, while the GHC -O +RTS -G1 line is nice and smooth, which is what we want.
I don't know if making this change defeats the purpose of some tests, which is why I'm opening this ticket.
Trac metadata
| Trac field | Value |
|---|---|
| Version | 7.10.1 |
| Type | Task |
| TypeOfFailure | OtherFailure |
| Priority | normal |
| Resolution | Unresolved |
| Component | Test Suite |
| Test case | |
| Differential revisions | |
| BlockedBy | |
| Related | |
| Blocking | |
| CC | ezyang, thoughtpolice |
| Operating system | |
| Architecture |