Skip to content
  • Ben Gamari's avatar
    testsuite: Ensure that makefile tests get run · b15a7fb8
    Ben Gamari authored and Marge Bot's avatar Marge Bot committed
    Previously `makefile_test` and `run_command` tests could easily end up
    in a situation where they wouldn't be run if the user used the
    `only_ways` modifier. The reason is to build the set of a ways to run
    the test in we first start with a candidate set determined by the test
    type (e.g. `makefile_test`, `compile_run`, etc.) and then filter that
    set with the constraints given by the test's modifiers.
    
    `makefile_test` and `run_command` tests' candidate sets were simply
    `{normal}`, and consequently most uses of `only_ways` would result in
    the test being never run.
    
    To avoid this we rather use all ways as the candidate sets for these
    test types. This may result in a few more testcases than we would like
    (given that some `run_command` tests are insensitive to way) but this
    can be fixed by adding modifiers and we would much rather run too many
    tests than too few.
    
    This fixes #16042 and a number of other tests afflicted by the same issue.
    However, there were a few cases that required special attention:
    
     * `T14028` is currently failing and is therefore marked as broken due
       to #17300
    
     * `T-signals-child` is fragile in the `threaded1` and `threaded2` ways
       (tracked in #17307)
    b15a7fb8