[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: "Test results"
Sorry for the long dormancy on SRFI-64. I'll post an update soon.
But first I'd like some feedback on an API for "test results":
A test runner maintains a set of "result properties" associated with
the current or most recent test. (I.e. the properties of the
most recent test are available as long as a new test hasn't started.)
Each property has a name (a symbol) and a value (any value).
Some properties are standard or set by the implementation.
(test-result-ref [runner] 'pname [default])
Returns the property value associated with the pname property name.
If there is no value assocate with 'pname return default,
or #t if default isn't specified.
(test-result-set! [runner] 'pname value)
Sets the property value associated with the pname property name to value.
(test-result-clear [runner] ['pname])
If 'pname is specified, remove the property with the name 'pname.
If 'pname is not specified, remove all result properties.
The implementation automatically calls test-result-clear
at the start of a test-assert and similar procedures.
Returns an association list of the current result properties.
It is unspecified if the result shares state with the test-runner.
The result should not be modified; on the other hand the result
may be inclicitly modified by future test-result-set! or
test-result-clear calls, with one exception: A test-result-clear
without a 'pname doesn't not modify the returned alist. Thus you
can "archive" result objects from previous runs.
Issue: Or should we just say that test-result-alist returns a
We also need a list of standard properties:
One of 'pass, 'xpass, 'fail, 'xfail, 'skip
Location of test-assert etc in test suite source code.
The source form, if meaningful/available.
The expected result, if meaningful/available.
The actual result, if meaningful/available.
The test-runner call-back functions (such as that bound using
test-runner-on-test) is modified to only take a test-runner,
rather than also take an explicit test-result, since the latter
is now part of the former.