[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: Meta-test suite

This page is part of the web mail archives of SRFI 64 from before July 7th, 2015. The new archives for SRFI 64 contain all messages, not just those from before July 7th, 2015.



On Wed, 9 Mar 2005, Per Bothner wrote:

> Donovan Kolbly wrote:
> > I've started working on a meta-test suite for SRFI-64.
> 
> Cool!  It would be nice to incorporate such a test into SRFI-64 itself,
> if you agree.

That would be great by me.  I've always thought that test suites should be
a standard part of SRFIs, and now, with SRFI-64, we have the foundation to
start doing so.  What better place to start?

> 
> I'm really overloaded with various things, but I'll try it out asap.

That'd be cool, if you get a chance.  The latest one tickles some
ambiguities in the spec, too.

> 
> I'd alse like to see the test-runner you used to generate the web page.
> Is that portable?  I assume it should be possible to make a portable
> version, by just display-ing html directly.  (I can see the line
> number links might not be portable.)  If so, it might be nice
> to also make that available from the SRFI document, at some point.

It's not portable at all, although that would be nice.

Currently, it's generated in two steps.  I have a test-runner which
generates an XML summary (using SXML internally and then writing out XML)  
[source code:  
<http://axis.rscheme.org:7973/src/file/view/lib-1/HEAD/srfi/64/xmlrunner.scm>]
and then the web server converts it into HTML on the fly (again, mostly
SXML crunching, but there also using RScheme's xpath facility).

As you say, the line-number stuff is highly non-portable (I even had to 
drop a quick hack in RScheme to get the line # that a form *ends* on), but 
the runner in theory could just extract that using the test results alist.  

More crucially, there does not seem to be any way for a portable runner to
keep track of the nesting of test suites.  I'm not sure yet what to
suggest for SRFI-64 to track suite nesting, but something along the lines
of a `test-runner-on-suite-begin!' and `test-runner-on-suite-end!' would
probably do it (*).

In my implementation, the default runner keeps track of all results all
the time, complete with nesting structure, and so when I'm dumping the
results it's easy to walk the test-result tree using my internal data
structure.


[*] do you want to call these things "suites" or "groups"?  The
    terminology seems to vary from place to place, or am I still missing
    something....


-- 
-- Donovan Kolbly                    (  d.kolbly@xxxxxxxxxxx
				     (  http://www.rscheme.org/~donovan/