[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: "Test results"
On Tue, 8 Mar 2005, Per Bothner wrote:
> Donovan Kolbly wrote:
> > The section "Test results" talks about what test results are, but it isn't
> > immediately clear how they are obtained as association lists. For my
> > instance, I am implementing a <test-result> class, and I suppose I will
> > need to roll up the fields into an alist. I can do that... but when?
> The test-runner-on-test/test-runner-on-test! parameter is part of the API,
> and the function returned from (test-runner-on-test runner) takes a
> test-result association list, so you have to create it by then.[...]
Aha, apologies, I see it. I didn't notice it because it didn't use the
key word "result".
> I'm not wedded to using alists - I used them as a simple extensible
> API. We could instead hide the current test result in the test-runner,
> and use callback functions:
> (test-result-get 'property-name [runner])
> (test-result-set! 'property-name value [runner])
> (test-result-clear [runner])
> The reference implementation can use a "private" association list stored
> in the test-runner. RScheme can use a <test-result> object.
I haven't implemented custom runners yet, but I don't think it'll be a
problem to use alists. I should be able to coerce it just before I call
the custom on-test procedure.
Although, that doesn't work if we allow the semantics of
"test-result-set!". If something like that is needed, then the
abstraction is preferable.
I'd even prefer clarifying when you can call a mutator: by default, I
wouldn't allow side-effects except in the extent of a custom runner hook
(as set with test-runner-on-test!), because I may want to export the
results out of the system when the custom runner completes.
> We might add some convenience functions:
> (test-result-passed?) == (memq (test-result-get 'result-kind) '(PASS XPASS))
> This also solves Alex Shinn's request for a way to query the result of a
> test. I'm assuming test-result-clear gets called at the start of a test-*,
> so the previous test's properties are available until then:
> (test-assert foo)
> (if (not (test-result-passed?))
> (test-skip ....))
I can certainly make read access to the most recent results work. I need
to think about write access.
-- Donovan Kolbly ( d.kolbly@xxxxxxxxxxx