This page is part of the web mail archives of SRFI 78 from before July 7th, 2015. The new archives for SRFI 78 contain all messages, not just those from before July 7th, 2015.
Thien-Thi Nguyen wrote:
Per Bothner <per@xxxxxxxxxxx> writes: >>Having a standard API makes this easier.but having a standard external (text) test report format is best.
Useful, yes, but not as importat as the API for writing test cases. What one needs is a mechanism to so one notices and can investigate if there is a change in test results. A simple 'diff' will do that, in a pinch. But the test-runner framework does allow you to customize the test report format. The output format of the default "simple" test-runner is based on what I'm used to from gcc and other dejagnu-based test suites: $ make check-num kawa -f "/home/bothner/Kawa/head/testsuite/num-test.scm" %%%% Starting test numbers (Writing full log to "numbers.log") /home/bothner/Kawa/head/testsuite/num-test.scm:3: FAIL # of expected passes 1663 # of unexpected failures 1 The "numbers.log" file has the complete test results in gory detail. The line number in the FAIL line is nice but it does admittedly have the problem that not all implementations can provide it. The detailed log: %%%% Starting test numbers Group begin: numbers Test begin: source-file: "/home/bothner/Kawa/head/testsuite/num-test.scm" source-line: 3 source-form: (test-approximate 1.5 (sqrt 2) 0.02) Test end: result-kind: fail actual-value: 1.4142135623730951 expected-value: 1.5 Test begin: source-file: "/home/bothner/Kawa/head/testsuite/num-test.scm" source-line: 4 source-form: (test-eqv 9223372036854775808 (- (- (expt 2 63)))) Test end: result-kind: pass actual-value: 9223372036854775808 expected-value: 9223372036854775808 .... Group end # of expected passes 1663 # of unexpected failures 1 [The failure is because I deliberately changed the expected value 1.4 to 1.5.] -- --Per Bothner per@xxxxxxxxxxx http://per.bothner.com/