The executable is unused so it shouldn't need to be built and
cause problems.

Also, convert the test to use the testsuite