OK. I believe @mpickering knows all of the parts of the code that need to be changed to add benchmark runs to head.hackage (we do something similar for bytestring/containers/text), but I'm not sure if these steps are documented anywhere.
I'm confused. Wasn't the thing that triggered ghc#23222 (closed) the benchmark suite for atomic-counter, not the test suite? (We could still add the test suite, I suppose, but it's a separate thing entirely.)