[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: proosal: removing test-result objects

This page is part of the web mail archives of SRFI 64 from before July 7th, 2015. The new archives for SRFI 64 contain all messages, not just those from before July 7th, 2015.

On Sun, 3 Apr 2005, Per Bothner wrote:

> As discussed in past messages, I'm leaning towords removing the
> concept of a test-result "value", which in the current specification
> is an association list.  Instead, I think it is better to make the
> "most recent test-result" be part of the test-runner, and allow
> extra properties there.
> Specifically, we add:
> ;; Get a named property - #f if unknown.
> (test-result [runner] 'property-name)
> These are properties of the current or most recent test.
> Exampes:
> (test-result 'kind) -> one of 'pass, 'xpass, 'fail, 'xfail, or 'skip.
> (test-passed?) -> same as (memq (test-result 'kind) '(pass xpass))
> (test-result 'source-line) -> line number if known or #f otherwise
> (test-result 'source-filename) -> file name known or #f otherwise
> (test-result 'source-form) -> test expresion as a list or #f
> (test-result 'test-name) -> test name or #f
> (test-result-set! [runner] 'property-name value) ;; set property
> (test-result-clear! [runner])
> This is called internally at the start of a test to reset the
> test-result state.  Specifically, all properties set by
> test-result-set! are cleared.
> The function on-test in (test-runner-on-test! runner on-test)
> is now a 1-argument function that only takes a test-runner.
> It gets the "test-result" from the test-runner.
> Note that both test-cases and test-runner can examine these
> properties.  For example (test-passed?) can be used to decide
> whether to skip subsequent tests.
> I also thing it simplies the implementation and api.  We can
> set properties at various statges during the execution of a test.
> Comments?  Does this seem like an improvement?

I like the idea of being able to programmatically decide wether to
continue, for example.  Then you could, e.g., define a custom runner that,
say, tolerates at most 10 errors and then skips everything else in the
rest of the group.

In some ways it seems better, although I don't quite get why you want to
*remove* test-result objects.  My mental model of test suites is that it's
a tree whose root is the suite, whose interior nodes are groups, and whose
leaves are test cases (each test case being represented by a test result).  
I guess you have the same basic model, but none of those objects are first
class :-)

So I'd probably go the other way and make test results (or, more likely,
test cases) first class, something like so:

  (test-case [runner])  ->  current or most recent test case, or #f if none

  (test-case-kind tcase) -> one of (pass xpass ...) [*]
  (test-case-passed? tcase) ; same as (memq (test-case-kind tcase) '(pass xpass))
  (test-case-property tcase 'property [value])   ; property setter/getter
  (test-case-name tcase) -> test name 

[* although 'kind' is looking like a strange name here; maybe 

-- Donovan Kolbly                    (  d.kolbly@xxxxxxxxxxx
				     (  http://www.rscheme.org/~donovan/