[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: "Test results"

Donovan Kolbly wrote:
On Mon, 15 Aug 2005, Per Bothner wrote:

A test runner maintains a set of "result properties" associated with
the current or most recent test.  (I.e. the properties of the
most recent test are available as long as a new test hasn't started.)

Each property has a name (a symbol) and a value (any value).
Some properties are standard or set by the implementation.

We should clarify the semantics of attempting to set a standard property. For example, is it legal to set the 'kind property?

I would say that is undefined.

Generally properties would be set by the implementation,
and read by the test-runner.  In some cases a test-runner
might add some extra properties, but I expect that to be
rare - offhand I don't see any use case for that.

Would that cause test-runner-pass-count and friends to be adjusted?


What about setting the 'kind property before the test has completed?

And anyway, what is the value of the 'kind property for a test that is in progress? I supposed it would just not be set.


>> (test-result-ref [runner] 'pname [default])

> Hmm.  I think statically ambiguous interfaces are confusing.

> If you see the following (admittedly poorly written) code fragment:

>   (test-result-ref x y)

you will need much more information (perhaps located arbitrarily far away in the program) to understand what is actually being done. Is the property denoted by y of runner x being accessed, or is property x of the current runner being accessed, with a default value of y?

We can pick one. I'd say here x is the runner and y the 'pname.

Perhaps we should just make these [runner] arguments required.
It's important to have a convenient/terse syntax for test suites,
but there is little value in making test-runner code maximally terse.

>> Returns the property value associated with the pname property name.
>> If there is no value assocate with 'pname return default,
>> or #t if default isn't specified.

> Why #t? I would think that #f, as the Most Distinguished Value, is a > more useful default default.

That was a typo.  I meant to write #f.
	--Per Bothner
per@xxxxxxxxxxx   http://per.bothner.com/