[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: "Test results"

This page is part of the web mail archives of SRFI 64 from before July 7th, 2015. The new archives for SRFI 64 contain all messages, not just those from before July 7th, 2015.

Donovan Kolbly wrote:
The section "Test results" talks about what test results are, but it isn't
immediately clear how they are obtained as association lists.  For my
instance, I am implementing a <test-result> class, and I suppose I will
need to roll up the fields into an alist.  I can do that... but when?

The test-runner-on-test/test-runner-on-test! parameter is part of the API,
and the function returned from (test-runner-on-test runner) takes a
test-result association list, so you have to create it by then.

The test-%report-result function also expects an alist, but you're free
to re-implement that function as its not part of the API.

I'm not wedded to using alists - I used them as a simple extensible
API.  We could instead hide the current test result in the test-runner,
and use callback functions:
  (test-result-get 'property-name [runner])
  (test-result-set! 'property-name value [runner])
  (test-result-clear [runner])
The reference implementation can use a "private" association list stored
in the test-runner.  RScheme can use a <test-result> object.

We might add some convenience functions:
  (test-result-passed?) == (memq (test-result-get 'result-kind) '(PASS XPASS))

This also solves Alex Shinn's request for a way to query the result of a
test.  I'm assuming test-result-clear gets called at the start of a test-*,
so the previous test's properties are available until then:

   (test-assert foo)
   (if (not (test-result-passed?))
      (test-skip ....))

Also, I don't understand the sentence "The test-kind association return
one of the following symbols".  Does that mean that one key in the alist
is the symbol 'test-kind and the value is one of the indicated symbols,
so, for example, the following might be a test result:

   ((test-kind . PASS) (source-file . "foo.scm") (source-line . 13))


(I notice that I'm inconsistent about whether result kinds are upper-case or
lower-case, which makes a difference in case-sensitive Schemes.  Oops.)
	--Per Bothner
per@xxxxxxxxxxx   http://per.bothner.com/