It appears that specifying only_ways(llvm_ways) for tests is not sufficient. E.g. when looking for the test T8131 in the llvm-CI-runner's logs, you won't find it. It is written like this:

test('T8131', [cmm_src, only_ways(llvm_ways)], compile_fail, [''])

The only optllvm-way tests are on validate-aarch64-linux-deb9

=====> T5681(optllvm) 2409 of 6747 [0, 83, 8]
cd "llvm/should_compile/T5681.run" &&  "/builds/ghc/ghc/inplace/bin/ghc-stage2" -c T5681.hs -dcore-lint -dstg-lint -dcmm-lint -no-user-package-db -rtsopts -fno-warn-missed-specialisations -fshow-warning-groups -fdiagnostics-color=never -fno-diagnostics-show-caret -Werror=compat -dno-debug-output  -O -fllvm 
=====> T6158(optllvm) 2410 of 6747 [0, 83, 8]
cd "llvm/should_compile/T6158.run" &&  "/builds/ghc/ghc/inplace/bin/ghc-stage2" -c T6158.hs -dcore-lint -dstg-lint -dcmm-lint -no-user-package-db -rtsopts -fno-warn-missed-specialisations -fshow-warning-groups -fdiagnostics-color=never -fno-diagnostics-show-caret -Werror=compat -dno-debug-output  -O -fllvm -package vector -package primitive
=====> T7571(optllvm) 2411 of 6747 [0, 83, 8]
cd "llvm/should_compile/T7571.run" &&  "/builds/ghc/ghc/inplace/bin/ghc-stage2" -c T7571.cmm -dcore-lint -dstg-lint -dcmm-lint -no-user-package-db -rtsopts -fno-warn-missed-specialisations -fshow-warning-groups -fdiagnostics-color=never -fno-diagnostics-show-caret -Werror=compat -dno-debug-output  -O -fllvm  -no-hs-main 
=====> T8131b(optllvm) 2413 of 6747 [0, 83, 8]
cd "llvm/should_compile/T8131b.run" &&  "/builds/ghc/ghc/inplace/bin/ghc-stage2" -c T8131b.hs -dcore-lint -dstg-lint -dcmm-lint -no-user-package-db -rtsopts -fno-warn-missed-specialisations -fshow-warning-groups -fdiagnostics-color=never -fno-diagnostics-show-caret -Werror=compat -dno-debug-output  -O -fllvm 
=====> T11649(optllvm) 2414 of 6747 [0, 83, 8]

These are triggered by the magic incantation:

def f( name, opts ):
  opts.only_ways = ['optllvm', 'llvm', 'debugllvm']

setTestOpts(f)

validate-x86_64-linux-deb9-llvm won't even test those...