... | ... | @@ -79,7 +79,7 @@ For adding any test case, follow these guide lines and then refer to the more sp |
|
|
|
|
|
Below we will look at some of the more common test case setups.
|
|
|
|
|
|
## A single module test case
|
|
|
## 1. A single module test case
|
|
|
|
|
|
|
|
|
A single module test case is very easy. Simply name the Haskell source files the same as your test
|
... | ... | @@ -109,7 +109,7 @@ test('drvfail001', normal, compile_fail, ['']) |
|
|
|
|
|
For more detailed control of a test case, see below. \\REF
|
|
|
|
|
|
## A multiple module test case
|
|
|
## 2. A multiple module test case
|
|
|
|
|
|
|
|
|
A multiple module test case is slightly more complex then a single module one. Firstly we a concerned with how to handle the simplest form of a multiple module test case, that is one where the whole test case can be built in one go using the `--make` command of GHC. If you have more complex needs (like compiling source files that `--make` can't handle, and/or need to compile different modules with different GHC arguments, then see below)
|
... | ... | @@ -144,7 +144,7 @@ test('Over', |
|
|
['OverD', '-no-hs-main -c -v0'])
|
|
|
```
|
|
|
|
|
|
## Advanced multiple module test case
|
|
|
## 3. Advanced multiple module test case
|
|
|
|
|
|
|
|
|
If you have a test case that can't be built with the simpler two methods described above then you should try the method described below. The build method below allows you to explicitly provide a list of `(source file, GHC flags)` tuples. GHC then builds those in the order you specify. This is useful for test cases say that use a .cmm source file or .c source file, these are files that GHC can build but aren't picked up by `--make`.
|
... | ... | @@ -184,9 +184,7 @@ test('Check01', normal, multi_compile_fail, ['Check01', [ |
|
|
|
|
|
This test case must use the `multi_compile_fail` method as it relies on being able to compile the file Check01_B.hs with the argument '-trust base' but not compile any of the other files with this flag.
|
|
|
|
|
|
##
|
|
|
|
|
|
## Format of the test entries in all.T
|
|
|
## 4. Format of the test entries in all.T
|
|
|
|
|
|
|
|
|
Each test in a `test.T` file is specified by a line the form
|
... | ... | @@ -198,11 +196,11 @@ test(<name>, <setup>, <test-fn>, <args...>) |
|
|
|
|
|
Where \<args...\> is a list of arguments.
|
|
|
|
|
|
### The \<name\> field
|
|
|
### 4.1 The \<name\> field
|
|
|
|
|
|
*\<name\>* is the name of the test, in quotes (' or ").
|
|
|
|
|
|
### The \<setup\> field
|
|
|
### 4.2 The \<setup\> field
|
|
|
|
|
|
*\<setup\>* is a function (i.e. any callable object in Python) which allows the options for this test to be changed.
|
|
|
There are many pre-defined functions which can be used in this field:
|
... | ... | @@ -326,7 +324,7 @@ For example, to expect an exit code of 3 and omit way 'opt', we could use |
|
|
|
|
|
as the `<setup>` argument.
|
|
|
|
|
|
### Performance tests
|
|
|
### 4.3 Performance tests
|
|
|
|
|
|
|
|
|
Performance tests have recently been revamped significantly and are now much easier to use.
|
... | ... | @@ -388,7 +386,7 @@ In summary: |
|
|
|
|
|
See [Running Performance Tests](building/running-tests/performance-tests) on how to run these tests.
|
|
|
|
|
|
### The \<test-fn\> field
|
|
|
### 4.4 The \<test-fn\> field
|
|
|
|
|
|
*\<test-fn\>* is a function which describes how the test should be
|
|
|
built and maybe run. It also determines the number of arguments for
|
... | ... | @@ -477,7 +475,7 @@ file. The possible test functions are: |
|
|
no argument is given, use the name of the test as the target.
|
|
|
Works like **run_command** otherwise.
|
|
|
|
|
|
## Adding tests with external dependencies
|
|
|
## 5. Adding tests with external dependencies
|
|
|
|
|
|
If you test has non-boot dependencies then it can't be added directly to the GHC tree.
|
|
|
The alternative is to add the test to [`head.hackage`](https://gitlab.haskell.org/ghc/head.hackage), there the test can depend on any libraries you want but failures won't stop merges, they will only be picked up later.
|
... | ... | @@ -486,7 +484,7 @@ These tests are primarily suited for tests generated using quickcheck or other r |
|
|
libraries.
|
|
|
|
|
|
|
|
|
## Sample output files
|
|
|
## 6. Sample output files
|
|
|
|
|
|
|
|
|
Normally, the sample `stdout` and `stderr` for a test T go in the
|
... | ... | @@ -506,7 +504,7 @@ the current configuration will be selected; for example if the |
|
|
platform is `i386-unknown-mingw32` then `T.stderr-i386-unknown-mingw32`
|
|
|
will be picked in preference to `T.stderr`.
|
|
|
|
|
|
## Threaded Considerations
|
|
|
## 7. Threaded Considerations
|
|
|
|
|
|
|
|
|
The testsuite has fairly good support for running tests in parallel using a thread pool of size specified by the `THREADS=<value>`. This does mean you need to be careful when writing test cases to keep them independent of each other. You are usually not able to share files between test cases as they can run in arbitrary order and will easily conflict with each other. If you must write test cases that are dependent on each other, be sure to use the `high_memory_usage` setup function that insures a test case runs by itself in the main testsuite thread. All dependent test cases should use the `high_memory_usage` setup function. Try not to do this extensively though as it means we can't easily speed up the testsuite by throwing cores at it.
|
... | ... | |