Commit ca535f95 authored by Tamar Christina's avatar Tamar Christina

testsuite: allow accepting of fine grained results [skip ci]

Summary:
Sometimes we need to be able to mass accept changes that are platform
specific and that can't be normalized away using our string formatters.

e.g. differences in I/O manager errors or behaviors. This allows one
to do so easier than before and less error prone.

I have updated the docs and made it clear this should only be used
when a normalizer won't work:

https://ghc.haskell.org/trac/ghc/wiki/Building/RunningTests/Updating

Test Plan: Manually tested while working on new I/O manager

Reviewers: bgamari

Subscribers: thomie, carter

Differential Revision: https://phabricator.haskell.org/D4549
parent 4de585a5
# #
# (c) Simon Marlow 2002 # (c) Simon Marlow 2002
# #
...@@ -9,7 +9,7 @@ ...@@ -9,7 +9,7 @@
# variable config below. The fields of the structure are filled in by # variable config below. The fields of the structure are filled in by
# the appropriate config script(s) for this compiler/platform, in # the appropriate config script(s) for this compiler/platform, in
# ../config. # ../config.
# #
# Bits of the structure may also be filled in from the command line, # Bits of the structure may also be filled in from the command line,
# via the build system, using the '-e' option to runtests. # via the build system, using the '-e' option to runtests.
...@@ -28,6 +28,8 @@ class TestConfig: ...@@ -28,6 +28,8 @@ class TestConfig:
# Accept new output which differs from the sample? # Accept new output which differs from the sample?
self.accept = 0 self.accept = 0
self.accept_platform = 0
self.accept_os = 0
# File in which to save the summary # File in which to save the summary
self.summary_file = '' self.summary_file = ''
...@@ -70,7 +72,7 @@ class TestConfig: ...@@ -70,7 +72,7 @@ class TestConfig:
# Flags we always give to this compiler # Flags we always give to this compiler
self.compiler_always_flags = [] self.compiler_always_flags = []
# Which ways to run tests (when compiling and running respectively) # Which ways to run tests (when compiling and running respectively)
# Other ways are added from the command line if we have the appropriate # Other ways are added from the command line if we have the appropriate
# libraries. # libraries.
...@@ -109,7 +111,7 @@ class TestConfig: ...@@ -109,7 +111,7 @@ class TestConfig:
# the timeout program # the timeout program
self.timeout_prog = '' self.timeout_prog = ''
self.timeout = 300 self.timeout = 300
# threads # threads
self.threads = 1 self.threads = 1
self.use_threads = 0 self.use_threads = 0
......
...@@ -1597,7 +1597,17 @@ def compare_outputs(way, kind, normaliser, expected_file, actual_file, ...@@ -1597,7 +1597,17 @@ def compare_outputs(way, kind, normaliser, expected_file, actual_file,
if_verbose(1, 'Test is expected to fail. Not accepting new output.') if_verbose(1, 'Test is expected to fail. Not accepting new output.')
return 0 return 0
elif config.accept and actual_raw: elif config.accept and actual_raw:
if_verbose(1, 'Accepting new output.') if config.accept_platform:
if_verbose(1, 'Accepting new output for platform "'
+ config.platform + '".')
expected_path += '-' + config.platform
elif config.accept_os:
if_verbose(1, 'Accepting new output for os "'
+ config.os + '".')
expected_path += '-' + config.os
else:
if_verbose(1, 'Accepting new output.')
write_file(expected_path, actual_raw) write_file(expected_path, actual_raw)
return 1 return 1
elif config.accept: elif config.accept:
...@@ -1914,7 +1924,7 @@ def in_srcdir(name, suffix=''): ...@@ -1914,7 +1924,7 @@ def in_srcdir(name, suffix=''):
# Finding the sample output. The filename is of the form # Finding the sample output. The filename is of the form
# #
# <test>.stdout[-ws-<wordsize>][-<platform>] # <test>.stdout[-ws-<wordsize>][-<platform>|-<os>]
# #
def find_expected_file(name, suff): def find_expected_file(name, suff):
basename = add_suffix(name, suff) basename = add_suffix(name, suff)
......
...@@ -14,6 +14,9 @@ ...@@ -14,6 +14,9 @@
# CONFIG -- use a different configuration file # CONFIG -- use a different configuration file
# COMPILER -- select a configuration file from config/ # COMPILER -- select a configuration file from config/
# THREADS -- run n tests at once # THREADS -- run n tests at once
# PLATFORM -- if accepting a result, accept it for the current platform.
# OS -- if accepting a result, accept it for all wordsizes of the
# current os.
# #
# ----------------------------------------------------------------------------- # -----------------------------------------------------------------------------
...@@ -280,6 +283,15 @@ endif ...@@ -280,6 +283,15 @@ endif
ifeq "$(accept)" "YES" ifeq "$(accept)" "YES"
setaccept = -e config.accept=1 setaccept = -e config.accept=1
ifeq "$(PLATFORM)" "YES"
setaccept += -e config.accept_platform=1
endif
ifeq "$(OS)" "YES"
setaccept += -e config.accept_os=1
endif
else else
setaccept = setaccept =
endif endif
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment