Re: gratuitous optimization and benchmarking

["Taylor R. Campbell" <campbell@xxxxxxxxxx>]

I didn't see how you loaded the code in any of the systems, so I can't
tell where the disparity is exactly, but here are my guesses:

  1. In Scheme48, you probably neither set inline-values at the REPL
     nor used the module system.  By default, the REPL will compile
     code under the assumption that any binding can be redefined,
     including standard R5RS ones, so it inhibits the compiler from
     performing *any* kind of analyses on standard bindings.  This
     means that it was given no chance to recognize CWV, and that it
     must generate full, heap-allocated closures for the operands.
     Either use the module system or type this at the REPL first:

        ,set inline-values

  2. I can't explain why MzScheme is slower because I haven't done any
     work with it myself on this, but it would not surprise me to hear
     that MzScheme's compiler also needs guarantees provided by the
     module system and not the default REPL in order to recognize CWV.

  3. Chicken's compiler does not appear to make any effort at all to
     recognize CWV, and so the generated code is extremely similar in
     the way it allocates closures, except that there is an
     out-of-line call to C_call_with_values in TEST-V(V), which
     involves a lot more effort than a direct procedure call, as in
     TEST-M(M), particularly since the direct procedure call works
     with known arities, whereas C_call_with_values &c. dispatch on
     the arity.

For the record, this is how I ran the code:

T: no special command-line options
  (comfile 'soo)
  (load 'soo)
  (block (gc) (time (test-m)))
  (block (gc) (time (test-v)))
  (block (gc) (time (test-mm)))
  (block (gc) (time (test-vv)))

Scheme48: no special command-line options
  ,set inline-values
  ,load soo.scm
  ,collect
  ,time (test-m)
  ,collect
  ,time (test-v)
  ,collect
  ,time (test-mm)
  ,collect
  ,time (test-vv)

Gauche: gosh -fcase-fold
  (use srfi-19)
  (define (time thunk)
    (let ((t0 (current-time)))
      (thunk)
      (let ((t1 (current-time)))
        (let ((delta (time-difference t1 t0)))
          (+ (time-second delta)
             (* .00000001 (time-nanosecond delta)))))))
  (load "./soo")
  (begin (gc) (time test-m))
  (begin (gc) (time test-v))
  (begin (gc) (time test-mm))
  (begin (gc) (time test-vv))