GitLab

Summary

Test suite produces strange errors/results when run with optimizations on (for later GHCs). Also, results vary from run to run.

Steps to reproduce

Download and install J.

git clone https://github.com/vmchale/j-hs.git
cd j-hs/
cabal test -w ghc-8.10.2
cabal test -w ghc-8.0.2
cabal test -w ghc-8.10.2 -O0

(revision 475755ca3d3dc5ce0557e42ef0df321457b8ebac?)

Then this gives:

J dl
  Performs calculation and has sensible output:             OK
  Reads back type in the environment:                       OK
  Reads a string:                                           OK
  Sends an array to J:                                      OK
  Uses J to perform a complex calculation:                  OK
  Writes strigns to J values:                               OK
  Uses J for something Haskell would have a hard time with: j-test: internal error: index out of bounds
CallStack (from HasCallStack):
  error, called at ./Test/Tasty/Ingredients/ConsoleReporter.hs:178:20 in tasty-1.3.1-268a33715c1414c235611995fa9c09de6e68f7c1c3aa3455f08d0451da953e2d:Test.Tasty.Ingredients.ConsoleReporter
Test suite j-test: FAIL

for GHC 8.10.2, then works fine with GHC 8.0.2 or when optimization is set to -O0.

Unfortunately I can't get a stack trace since compiling with --enable-profiling gives yet another failure mode, viz.

J dl
  Performs calculation and has sensible output:             OK
  Reads back type in the environment:                       OK
  Reads a string:                                           OK
  Sends an array to J:                                      OK
  Uses J to perform a complex calculation:                  OK
  Writes strigns to J values:                               OK
  Uses J for something Haskell would have a hard time with: OK

All 7 tests passed (0.00s)
Test suite j-test: FAIL
Test suite logged to:
/home/vanessa/programming/haskell/current/j/dist-newstyle/build/x86_64-linux/ghc-8.10.2/j-0.1.0.0/t/j-test/test/j-0.1.0.0-j-test.log

...which is puzzling/contradictory!

Expected behavior

I expect it to pass the test suite or at least for the results to be consistent.

I definitely don't expect to see a tasty bug in the stack trace since I don't think that code is buggy.

Environment

• GHC version used: 8.10.2, 8.8.4, 8.6.5, 8.4.4, 8.2.2, 8.0.2, 7.10.3

Optional:

• Operating System: Lubuntu 18.04
• System Architecture: x86_64