Skip to content

Fix intermittent hie002 failure

Vladislav Zavialov requested to merge wip/hie002-stabilize into master

hie002 is a performance test that used to fail unpredictably:

max_bytes_used Decrease from x86_64-linux-deb9-debug baseline @ HEAD~2:
    Expected    hie002 (normal) max_bytes_used: 1190923992.0 +/-20%
    Lower bound hie002 (normal) max_bytes_used:    952739193
    Upper bound hie002 (normal) max_bytes_used:   1429108791
    Actual      hie002 (normal) max_bytes_used:    726270784
    Deviation   hie002 (normal) max_bytes_used:        -39.0 %
peak_megabytes_allocated Decrease from x86_64-linux-deb9-debug baseline @ HEAD~2:
    Expected    hie002 (normal) peak_megabytes_allocated: 2538.0 +/-20%
    Lower bound hie002 (normal) peak_megabytes_allocated:   2030
    Upper bound hie002 (normal) peak_megabytes_allocated:   3046
    Actual      hie002 (normal) peak_megabytes_allocated:   1587
    Deviation   hie002 (normal) peak_megabytes_allocated:  -37.5 %
*** unexpected stat test failure for hie002(normal)

max_bytes_used and peak_megabytes_allocated are too unstable without careful control of the runtime configuration. We fix this by using a more predictable metric, bytes allocated.

Merge request reports