[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: Meta-test suite
On Wed, 9 Mar 2005, Per Bothner wrote:
> Donovan Kolbly wrote:
> > I've started working on a meta-test suite for SRFI-64.
> Cool! It would be nice to incorporate such a test into SRFI-64 itself,
> if you agree.
That would be great by me. I've always thought that test suites should be
a standard part of SRFIs, and now, with SRFI-64, we have the foundation to
start doing so. What better place to start?
> I'm really overloaded with various things, but I'll try it out asap.
That'd be cool, if you get a chance. The latest one tickles some
ambiguities in the spec, too.
> I'd alse like to see the test-runner you used to generate the web page.
> Is that portable? I assume it should be possible to make a portable
> version, by just display-ing html directly. (I can see the line
> number links might not be portable.) If so, it might be nice
> to also make that available from the SRFI document, at some point.
It's not portable at all, although that would be nice.
Currently, it's generated in two steps. I have a test-runner which
generates an XML summary (using SXML internally and then writing out XML)
and then the web server converts it into HTML on the fly (again, mostly
SXML crunching, but there also using RScheme's xpath facility).
As you say, the line-number stuff is highly non-portable (I even had to
drop a quick hack in RScheme to get the line # that a form *ends* on), but
the runner in theory could just extract that using the test results alist.
More crucially, there does not seem to be any way for a portable runner to
keep track of the nesting of test suites. I'm not sure yet what to
suggest for SRFI-64 to track suite nesting, but something along the lines
of a `test-runner-on-suite-begin!' and `test-runner-on-suite-end!' would
probably do it (*).
In my implementation, the default runner keeps track of all results all
the time, complete with nesting structure, and so when I'm dumping the
results it's easy to walk the test-result tree using my internal data
[*] do you want to call these things "suites" or "groups"? The
terminology seems to vary from place to place, or am I still missing
-- Donovan Kolbly ( d.kolbly@xxxxxxxxxxx