Skip to content

testsuite: Add mechanism to collect generic metrics

Matthew Pickering requested to merge wip/testsuite-generic-stats into master
  • Generalise the metric logic by adding an additional field which allows you to specify how to query for the actual value. Previously the method of querying the baseline value was abstracted (but always set to the same thing).

  • This requires rejigging how the stat collection works slightly but now it's more uniform and hopefully simpler.

  • Introduce some new "generic" helper functions for writing generic stats tests.

    • collect_size ( deviation, path ) Record the size of the file as a metric

    • stat_from_file ( metric, deviation, path ) Read a value from the given path, and store that as a metric

    • collect_generic_stat ( metric, deviation, get_stat ) Provide your own get_stat function, lambda way: <Int>, which can be used to establish the value of the metric.

    • collect_generic_stats ( get_stats ): Like collect_generic_stat but provide the whole dictionary of metric definitions.

      { metric: { deviation: action: lambda way: } }

  • Introduce two new "size" metrics for keeping track of build products.

    • size_hello - The size of hello.o from compiling hello.hs
    • libdir - The total size of the libdir folder.
  • Track the number of modules in the AST tests

    • CountDepsAst
    • CountDepsParser

This lays the infrastructure for #24191 #22256 #17129

Edited by Matthew Pickering

Merge request reports