Hole fits tests: are they perf tests or not?
In the testsuite, typecheck/should_compile/all.T
has this bit:
test('hard_hole_fits', # Testing multiple hole-fits with lots in scope for #16875
compile_timeout_multiplier(0.010), # 1 is 300s, 0.010 is 3s. Without hole-fits it takes 1s
compile, ['-fdefer-type-errors -fno-max-valid-hole-fits -package ghc'])
test('T16875', # Testing one hole-fit with a lot in scope for #16875
compile_timeout_multiplier(0.0015), #0.45s
compile, ['-fdefer-type-errors -fno-max-valid-hole-fits -package ghc'])
These lines declare two tests with an increased timeout. But my slightly ailing, middle-aged laptop (2016 vintage) on a non-optimized DEBUG compiler can't meet these deadlines. I'm OK with that. But I don't know how to proceed.
The key question: are these tests testing performance?
- If yes: then we should use our perf-testing setup to do so rather more robustly (and only on optimized builds).
- If no: then we should drastically increase the timeout multiplier.
cc @Tritlo, the Grand Master of valid hole fits.