testsuite: Add mechanism to collect generic metrics
-
Generalise the metric logic by adding an additional field which allows you to specify how to query for the actual value. Previously the method of querying the baseline value was abstracted (but always set to the same thing).
-
This requires rejigging how the stat collection works slightly but now it's more uniform and hopefully simpler.
-
Introduce some new "generic" helper functions for writing generic stats tests.
-
collect_size ( deviation, path ) Record the size of the file as a metric
-
stat_from_file ( metric, deviation, path ) Read a value from the given path, and store that as a metric
-
collect_generic_stat ( metric, deviation, get_stat ) Provide your own
get_stat
function,lambda way: <Int>
, which can be used to establish the value of the metric. -
collect_generic_stats ( get_stats ): Like collect_generic_stat but provide the whole dictionary of metric definitions.
{ metric: { deviation: action: lambda way: } }
-
-
Introduce two new "size" metrics for keeping track of build products.
-
size_hello
- The size ofhello.o
from compiling hello.hs -
libdir
- The total size of thelibdir
folder.
-
-
Track the number of modules in the AST tests
- CountDepsAst
- CountDepsParser
This lays the infrastructure for #24191 (closed) #22256 (closed) #17129