[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: "Test results"

This page is part of the web mail archives of SRFI 64 from before July 7th, 2015. The new archives for SRFI 64 contain all messages, not just those from before July 7th, 2015.

On Tue, 8 Mar 2005, Per Bothner wrote:

> Donovan Kolbly wrote:
> > The section "Test results" talks about what test results are, but it isn't
> > immediately clear how they are obtained as association lists.  For my
> > instance, I am implementing a <test-result> class, and I suppose I will
> > need to roll up the fields into an alist.  I can do that... but when?
> The test-runner-on-test/test-runner-on-test! parameter is part of the API,
> and the function returned from (test-runner-on-test runner) takes a
> test-result association list, so you have to create it by then.[...]

Aha, apologies, I see it.  I didn't notice it because it didn't use the
key word "result".

> I'm not wedded to using alists - I used them as a simple extensible
> API.  We could instead hide the current test result in the test-runner,
> and use callback functions:
>    (test-result-get 'property-name [runner])
>    (test-result-set! 'property-name value [runner])
>    (test-result-clear [runner])
> The reference implementation can use a "private" association list stored
> in the test-runner.  RScheme can use a <test-result> object.

I haven't implemented custom runners yet, but I don't think it'll be a
problem to use alists.  I should be able to coerce it just before I call
the custom on-test procedure.

Although, that doesn't work if we allow the semantics of 
"test-result-set!".  If something like that is needed, then the 
abstraction is preferable.

I'd even prefer clarifying when you can call a mutator: by default, I
wouldn't allow side-effects except in the extent of a custom runner hook
(as set with test-runner-on-test!), because I may want to export the
results out of the system when the custom runner completes.

> We might add some convenience functions:
>    (test-result-passed?) == (memq (test-result-get 'result-kind) '(PASS XPASS))

> This also solves Alex Shinn's request for a way to query the result of a
> test.  I'm assuming test-result-clear gets called at the start of a test-*,
> so the previous test's properties are available until then:
>     (test-assert foo)
>     (if (not (test-result-passed?))
>        (test-skip ....))

I can certainly make read access to the most recent results work.  I need 
to think about write access.

-- Donovan Kolbly                    (  d.kolbly@xxxxxxxxxxx
				     (  http://www.rscheme.org/~donovan/