Skip to content

Testsuite: compile_timeout_multiplier is fragile

I'm taking a sweep through the testsuite, trying to eliminate failures in a DEBUG compiler. But two tests are stymieing me: pmcheck/should_compile/T11{276,374}. Both of these use the compile_timout_multiplier option in their all.T entries. This option means that, instead of the normal 300s allotted to the test, these tests get only 3s. In my setup, with a DEBUG (read: unoptimized) stage2 compiler, these tests fail when I'm running the testsuite in parallel. Then, when I run the tests individually, they succeed.

This poses two problems:

  1. I can only see the failure when my machine is stressed, and it's disconcerting to have a failure appear and disappear depending on how I look.
  2. There is (sometimes) a failure in DEBUG mode.

What is the goal of compile_timeout_multiplier? If we're really testing performance (which #11276 (closed) and #11374 (closed) indicate), then these tests should be in perf/compiler and be "stat" tests, where some wibbling in the numbers is expected (and where DEBUG compilers are exempted). I would imagine that compile_timeout_multiplier should only ever get values greater than 1, when a non-performance test is so expensive to run that it takes more than 300s.

As a local solution here, I think these two tests should move to perf/compiler. Other pattern-match tests also have compile_timeout_multiplier, too, and I can move these as well. Any objections?

Trac metadata
Trac field Value
Version 8.4.3
Type Bug
TypeOfFailure OtherFailure
Priority normal
Resolution Unresolved
Component Compiler
Test case
Differential revisions
BlockedBy
Related
Blocking
CC
Operating system
Architecture
To upload designs, you'll need to enable LFS and have an admin enable hashed storage. More information