building/running-tests.md

@@ -177,38 +177,35 @@ these steps:
   regression tests go in the typechecker/ directory, parser tests
   go in parser/, and so on.  
 
-> >
-> > It's not always possible to find a single best place for a test;
-> > in those cases just pick one which seems reasonable.
+>
+> It's not always possible to find a single best place for a test;
+> in those cases just pick one which seems reasonable.
 
+>
+> Under each main directory may be up to three subdirectories:
+>
+> > **should_compile**:    
 > >
-> > Under each main directory may be up to three subdirectories:
-
-> > >
-> > > should_compile:
 > > >
-> > > >
-> > > > tests which need to compile only
-
-> > >
-> > > should_fail:    
-> > >
-> > > >
-> > > > tests which should fail to compile and generate a particular error message
-
+> > > tests which need to compile only
+> >
+> > **should_fail**:    
+> >
 > > >
-> > > should_run:
+> > > tests which should fail to compile and generate a particular error message
+> >
+> > **should_run**:
+> >
 > > >
-> > > >
-> > > > tests which should compile, run with some specific input, and generate a particular output.
+> > > tests which should compile, run with some specific input, and generate a particular output.
 
-> >
-> > We don't always divide the tests up like this, and it's not
-> > essential to do so (the directory names have no meaning as
-> > far as the test driver is concerned).        
+>
+> We don't always divide the tests up like this, and it's not
+> essential to do so (the directory names have no meaning as
+> far as the test driver is concerned).        
 
 1. Having found a suitable place for the test, give the test a name.
-  For regression tests, we often just name the test after the bug number (e.g. 2047).
+  For regression tests, we often just name the test after the bug number (e.g. T2047).
   Alternatively, follow the convention for the directory in which you place the
   test: for example, in typecheck/should_compile, tests are named
   tc001, tc002, and so on.  Suppose you name your test T, then
@@ -256,214 +253,197 @@ these steps:
         test(<name>, <setup>, <test-fn>, <args>)
   ```
 
-  where
+  The format of these fields is described in the [next section](building/running-tests#format-of-the-test-entries).
 
-> > *\<name\>* is the name of the test, in quotes (' or ").
 
-> > *\<setup\>*  is a function (i.e. any callable object in Python)
-> > which allows the options for this test to be changed.
-> > There are many pre-defined functions which can be
-> > used in this field:
+A multi-module test is straightforward.  It usually goes in a
+directory of its own (although this isn't essential), and the source
+files can be named anything you like.  The test must have a name, in
+the same way as a single-module test; and the stdin/stdout/stderr
+files follow the name of the test as before.  In the same directory,
+place a file 'test.T' containing a line like
 
-> > > **normal**                don't change any options from the defaults
+```wiki
+   test(multimod001, normal, multimod_compile_and_run, \
+                 [ 'Main', '-fglasgow-exts', '', 0 ])
+```
 
-> > > **skip**                  skip this test
 
-> > > **skip_if_no_ghci**       skip unless GHCi is available
+as described above.
 
-> > > **skip_if_fast**          skip if "fast" is enabled
 
-> > > **omit_ways(ways)**       skip this test for certain ways
+For some examples, take a look in tests/ghc-regress/programs.
 
-> > > **only_ways(ways)**       do this test certain ways only
+# Format of the test entries
 
-> > > **extra_ways(ways)**      add some ways which would normally be disabled
 
-> > > **omit_compiler_types(compilers)**                           skip this test for certain compilers
+Each test in a `test.T` file is specified by a line the form
 
-> > > **only_compiler_types(compilers)**       do this test for certain compilers only
+```wiki
+      test(<name>, <setup>, <test-fn>, <args>)
+```
 
-> > > **expect_broken(bug)** this test is a expected not to work due to the indicated trac bug number
+## The \<name\> field
 
-> > > **expect_broken_for(bug, ways)** as expect_broken, but only for the indicated ways
+*\<name\>* is the name of the test, in quotes (' or ").
 
-> > > **if_compiler_type(compiler_type, f)** Do `f`, but only for the given compiler type
+## The \<setup\> field
 
-> > > **if_platform(plat, f)**  Do `f`, but only if we are on the specific platform given
+*\<setup\>*  is a function (i.e. any callable object in Python)
+which allows the options for this test to be changed.
+There are many pre-defined functions which can be
+used in this field:
 
-> > > **if_tag(tag, f)**        do `f` if the compiler has a given tag
+- **normal**                don't change any options from the defaults
+- **skip**                  skip this test
+- **skip_if_no_ghci**       skip unless GHCi is available
 
-> > > **unless_tag(tag, f)**    do `f` unless the compiler has a given tag
+- **skip_if_fast**          skip if "fast" is enabled
 
-> > > **set_stdin(file)**       use a different file for stdin
+- **omit_ways(ways)**       skip this test for certain ways
 
-> > > **no_stdin**              use no stdin at all (otherwise use `/dev/null`)
+- **only_ways(ways)**       do this test certain ways only
 
-> > > **exit_code(n)**          expect an exit code of 'n' from the prog
+- **extra_ways(ways)**      add some ways which would normally be disabled
 
-> > > **extra_run_opts(opts)**  pass some extra opts to the prog
+- **omit_compiler_types(compilers)**                           skip this test for certain compilers
 
-> > > **no_clean**              don't clean up after this test
+- **only_compiler_types(compilers)**       do this test for certain compilers only
 
-> > > **extra_clean(files)**    extra files to clean after the test has completed
+- **expect_broken(bug)** this test is a expected not to work due to the indicated trac bug number
 
-> > > **reqlib(P)**             requires package P
+- **expect_broken_for(bug, ways)** as expect_broken, but only for the indicated ways
 
-> > > **req_profiling**         requires profiling
+- **if_compiler_type(compiler_type, f)** Do `f`, but only for the given compiler type
 
-> > > **ignore_output**         don't try to compare output
+- **if_platform(plat, f)**  Do `f`, but only if we are on the specific platform given
 
-> > > **alone**                 don't run this test in parallel with anything else
+- **if_tag(tag, f)**        do `f` if the compiler has a given tag
 
-> > > **literate**              look for a `.lhs` file instead of a `.hs` file
+- **unless_tag(tag, f)**    do `f` unless the compiler has a given tag
 
-> > > **c_src**                 look for a `.c` file
+- **set_stdin(file)**       use a different file for stdin
 
-> > > **cmd_prefix(string)**    prefix this string to the command when run
+- **no_stdin**              use no stdin at all (otherwise use `/dev/null`)
 
-> > > **normalise_slashes**     convert backslashes to forward slashes before comparing the output
+- **exit_code(n)**          expect an exit code of 'n' from the prog
 
-> >
-> > To use more than one modifier on a test, just put them in a list.
-> > For example, to expect an exit code of 3 and omit way 'opt', we could use
-> >
-> > ```wiki
-> >       [ omit_ways(['opt']), exit_code(3) ]
-> > ```
-> >
-> > >
-> > > as the `<setup>` argument.
+- **extra_run_opts(opts)**  pass some extra opts to the prog
 
-> >
-> > The following should normally not be used; instead, use the `expect_broken*`
-> > functions above so that the problem doesn't get forgotten about, and when we
-> > come back to look at the test later we know whether current behaviour is why
-> > we marked it as expected to fail:
+- **no_clean**              don't clean up after this test
 
-> > > **expect_fail**           this test is an expected failure, i.e. there is a known bug in the compiler, but we don't want to fix it.
+- **extra_clean(files)**    extra files to clean after the test has completed
 
-> > > **expect_fail_for(ways)** expect failure for certain ways 
+- **reqlib(P)**             requires package P
 
-> > *\<test-fn\>*
-> > is a function which describes how the test should be
-> > run, and determines the form of \<args\>.  The possible
-> > values are:
+- **req_profiling**         requires profiling
 
-> > >
-> > > compile
-> > >
-> > > >
-> > > > Just compile the program, the compilation should succeed.
+- **ignore_output**         don't try to compare output
 
-> > >
-> > > compile_fail
-> > >
-> > > >
-> > > > Just compile the program, the
-> > > > compilation should fail (error
-> > > > messages will be in T.stderr).
-> > > > This kind of failure is mandated by the language definition - it does **not** indicate any bug in the compiler.
+- **alone**                 don't run this test in parallel with anything else
 
-> > >
-> > > compile_and_run 
-> > >
-> > > >
-> > > > Compile the program and run it,
-> > > > comparing the output against the 
-> > > > relevant files.
+- **literate**              look for a `.lhs` file instead of a `.hs` file
 
-> > >
-> > > multimod_compile
-> > >
-> > > >
-> > > > Compile a multi-module program
-> > > > (more about multi-module programs
-> > > > below).
+- **c_src**                 look for a `.c` file
 
-> > >
-> > > multimod_compile_fail
-> > >
-> > > >
-> > > > Compile a multi-module program,
-> > > > and expect the compilation to fail
-> > > > with error messages in T.stderr.  This kind of failure does **not** indicate a bug in the compiler.
+- **cmd_prefix(string)**    prefix this string to the command when run
 
-> > >
-> > > multimod_compile_and_run
-> > >
-> > > >
-> > > > Compile and run a multi-module
-> > > > program.
+- **normalise_slashes**     convert backslashes to forward slashes before comparing the output
 
-> > >
-> > > compile_and_run_with_prefix
-> > >
-> > > >
-> > > > Same as compile_and_run, but with command to use to run the execution of the result binary.
 
-> > >
-> > > multimod_compile_and_run_with_prefix
-> > >
-> > > >
-> > > > Same as multimod_compile_and_run, but with command to use to run the execution of the result binary.
+The following should normally not be used; instead, use the `expect_broken*`
+functions above so that the problem doesn't get forgotten about, and when we
+come back to look at the test later we know whether current behaviour is why
+we marked it as expected to fail:
 
-> > >
-> > > run_command
-> > >
-> > > >
-> > > > Just run an arbitrary command.  The output is checked
-> > > > against `T.stdout` and `T.stderr` (unless `ignore_output`
-> > > > is used), and the stdin and expected exit code can be
-> > > > changed in the same way as for compile_and_run.
+- **expect_fail**           this test is an expected failure, i.e. there is a known bug in the compiler, but we don't want to fix it.
 
-> > >
-> > > ghci_script
-> > >
-> > > >
-> > > > Runs the current compiler, passing
-> > > > --interactive and using the specified
-> > > > script as standard input.
+- **expect_fail_for(ways)** expect failure for certain ways 
 
-> > *\<args\>*
-> > is a list of arguments to be passed to \<test-fn\>.
 
-> >
-> > For compile, compile_fail and compile_and_run, \<args\>
-> > is a list with a single string which contains extra
-> > compiler options with which to run the test.  eg.
-> >
-> > ```wiki
-> >     test('tc001', normal, compile, ['-fglasgow-exts'])
-> > ```
-> >
-> >
-> > would pass the flag -fglasgow-exts to the compiler
-> > when compiling tc001.
+To use more than one modifier on a test, just put them in a list.
+For example, to expect an exit code of 3 and omit way 'opt', we could use
 
-> >
-> > The multimod_ versions of compile and compile_and_run
-> > expect an extra argument on the front of the list: the
-> > name of the top module in the program to be compiled
-> > (usually this will be 'Main').
+```wiki
+      [ omit_ways(['opt']), exit_code(3) ]
+```
 
 
-A multi-module test is straightforward.  It usually goes in a
-directory of its own (although this isn't essential), and the source
-files can be named anything you like.  The test must have a name, in
-the same way as a single-module test; and the stdin/stdout/stderr
-files follow the name of the test as before.  In the same directory,
-place a file 'test.T' containing a line like
+as the `<setup>` argument.
+
+## The \<test-fn\> field
+
+*\<test-fn\>*
+is a function which describes how the test should be
+run, and determines the form of \<args\>.  The possible
+values are:
+
+- **compile**  Just compile the program, the compilation should succeed.
+
+- **compile_fail**
+  Just compile the program, the
+  compilation should fail (error
+  messages will be in T.stderr).
+  This kind of failure is mandated by the language definition - it does **not** indicate any bug in the compiler.
+
+- **compile_and_run**
+  Compile the program and run it,
+  comparing the output against the 
+  relevant files.
+
+- **multimod_compile**
+  Compile a multi-module program
+  (more about multi-module programs
+  below).
+
+- **multimod_compile_fail**
+  Compile a multi-module program,
+  and expect the compilation to fail
+  with error messages in T.stderr.  This kind of failure does **not** indicate a bug in the compiler.
+
+- **multimod_compile_and_run**
+  Compile and run a multi-module
+  program.
+
+- **compile_and_run_with_prefix**
+  Same as compile_and_run, but with command to use to run the execution of the result binary.
+
+- **multimod_compile_and_run_with_prefix**
+  Same as multimod_compile_and_run, but with command to use to run the execution of the result binary.
+
+- **run_command**
+  Just run an arbitrary command.  The output is checked
+  against `T.stdout` and `T.stderr` (unless `ignore_output`
+  is used), and the stdin and expected exit code can be
+  changed in the same way as for compile_and_run.
+
+- **ghci_script**
+  Runs the current compiler, passing
+  --interactive and using the specified
+  script as standard input.
+
+## The \<args\> field
+
+*\<args\>* is a list of arguments to be passed to \<test-fn\>.
+
+
+For compile, compile_fail and compile_and_run, \<args\>
+is a list with a single string which contains extra
+compiler options with which to run the test.  eg.
 
 ```wiki
-   test(multimod001, normal, multimod_compile_and_run, \
-                 [ 'Main', '-fglasgow-exts', '', 0 ])
+    test('tc001', normal, compile, ['-fglasgow-exts'])
 ```
 
 
-as described above.
+would pass the flag -fglasgow-exts to the compiler
+when compiling tc001.
 
 
-For some examples, take a look in tests/ghc-regress/programs.
+The multimod_ versions of compile and compile_and_run
+expect an extra argument on the front of the list: the
+name of the top module in the program to be compiled
+(usually this will be 'Main').
 
 # Sample output files