Skip to content

Absolute stat metrics are hard for a human to compare

Motivation

The absolute performance metrics emitted by CI suffer from a "wall of digits" effect:

                                              Baseline                       
                         Test    Metric          value      New value Change 
-----------------------------------------------------------------------------
          Conversions(normal) run/alloc       107088.0       107088.0  +0.0% 
           DeriveNull(normal) run/alloc    112050656.0    112050656.0  +0.0% 
     InlineArrayAlloc(normal) run/alloc   1600040712.0   1600040712.0  +0.0% 
 InlineByteArrayAlloc(normal) run/alloc   1440040712.0   1440040712.0  +0.0% 
InlineCloneArrayAlloc(normal) run/alloc   1600040872.0   1600040872.0  +0.0% 
          MethSharing(normal) run/alloc    480097864.0    480097864.0  +0.0% 
               T10359(normal) run/alloc       354344.0       354344.0  +0.0% 

The "baseline" and "new" values are hard to compare, and it is difficult to estimate their magnitude (number of digits).

Proposal

These numbers could be displayed using the "grouping characters" that are standard for showing numeric values in the current locale. For example, in US English, the display above could be rendered

                                                 Baseline
                         Test    Metric             value         New value Change 
----------------------------------------------------------------------------------
          Conversions(normal) run/alloc         107,088.0         107,088.0  +0.0% 
           DeriveNull(normal) run/alloc     112,050,656.0     112,050,656.0  +0.0% 
     InlineArrayAlloc(normal) run/alloc   1,600,040,712.0   1,600,040,712.0  +0.0% 
 InlineByteArrayAlloc(normal) run/alloc   1,440,040,712.0   1,440,040,712.0  +0.0% 
InlineCloneArrayAlloc(normal) run/alloc   1,600,040,872.0   1,600,040,872.0  +0.0% 
          MethSharing(normal) run/alloc     480,097,864.0     480,097,864.0  +0.0% 
               T10359(normal) run/alloc         354,344.0         354,344.0  +0.0% 

Different locales use different grouping characters; for example, some European locales use spaces instead of commas.

For platforms that support the Single Unix Specification (version 2 and higher) this might be as simple as adding a ' mark to the format specification passed to printf(3).

To upload designs, you'll need to enable LFS and have an admin enable hashed storage. More information