Re: "Test results"

[Per Bothner <per@xxxxxxxxxxx>]

Donovan Kolbly wrote:
> The section "Test results" talks about what test results are, but it isn't
> immediately clear how they are obtained as association lists.  For my
> instance, I am implementing a <test-result> class, and I suppose I will
> need to roll up the fields into an alist.  I can do that... but when?

The test-runner-on-test/test-runner-on-test! parameter is part of the API,
and the function returned from (test-runner-on-test runner) takes a
test-result association list, so you have to create it by then.

The test-%report-result function also expects an alist, but you're free
to re-implement that function as its not part of the API.

I'm not wedded to using alists - I used them as a simple extensible
API.  We could instead hide the current test result in the test-runner,
and use callback functions:
  (test-result-get 'property-name [runner])
  (test-result-set! 'property-name value [runner])
  (test-result-clear [runner])
The reference implementation can use a "private" association list stored
in the test-runner.  RScheme can use a <test-result> object.

We might add some convenience functions:
  (test-result-passed?) == (memq (test-result-get 'result-kind) '(PASS XPASS))

This also solves Alex Shinn's request for a way to query the result of a
test.  I'm assuming test-result-clear gets called at the start of a test-*,
so the previous test's properties are available until then:

   (test-assert foo)
   (if (not (test-result-passed?))
      (test-skip ....))

> Also, I don't understand the sentence "The test-kind association return
> one of the following symbols".  Does that mean that one key in the alist
> is the symbol 'test-kind and the value is one of the indicated symbols,
> so, for example, the following might be a test result:
>
>    ((test-kind . PASS) (source-file . "foo.scm") (source-line . 13))

Correct.

(I notice that I'm inconsistent about whether result kinds are upper-case or
lower-case, which makes a difference in case-sensitive Schemes.  Oops.)
--
	--Per Bothner
per@xxxxxxxxxxx   http://per.bothner.com/