Skip to content

Hole fits tests: are they perf tests or not?

In the testsuite, typecheck/should_compile/all.T has this bit:

test('hard_hole_fits', # Testing multiple hole-fits with lots in scope for #16875
     compile_timeout_multiplier(0.010), # 1 is 300s, 0.010 is 3s. Without  hole-fits it takes 1s
     compile, ['-fdefer-type-errors -fno-max-valid-hole-fits -package ghc'])
test('T16875',  # Testing one hole-fit with a lot in scope for #16875
     compile_timeout_multiplier(0.0015), #0.45s
     compile, ['-fdefer-type-errors -fno-max-valid-hole-fits -package ghc'])

These lines declare two tests with an increased timeout. But my slightly ailing, middle-aged laptop (2016 vintage) on a non-optimized DEBUG compiler can't meet these deadlines. I'm OK with that. But I don't know how to proceed.

The key question: are these tests testing performance?

  • If yes: then we should use our perf-testing setup to do so rather more robustly (and only on optimized builds).
  • If no: then we should drastically increase the timeout multiplier.

cc @Tritlo, the Grand Master of valid hole fits.

Edited by Richard Eisenberg
To upload designs, you'll need to enable LFS and have an admin enable hashed storage. More information