Re: Not quite enough abstraction

[Jussi Piitulainen <jpiitula@xxxxxxxxxxxxxxxx>]

David Rush writes:
> Jussi Piitulainen writes:
>> separate index arithmetic from backing vectors entirely? The nature
>> of the package would change rather a lot - quite possibly to the
>> better.
> I had the impression that we were mostly there right now. Or have
> you not added the index-object interfaces in the new revision? I
> could be seriously confused at this point.

I don't see what this has to do with the backing vectors. They are as
hidden as before. By "backing vector" I mean a one-dimensional data
structure that contains the actual elements of the array.

Yes, I've added a kind of index objects, so that the indices can be
packaged in a vector or an array, but they are just so that the
following three, for example, mean the same.

  (array-ref arr k1 ...)
  (array-ref arr (vector k1 ...))
  (array-ref arr (array (shape 0 n) k1 ...))

The mapping from k1 ... to the single integer index to the backing
vector is still hidden inside arr.

>> Where do your sparse matrices come from if not from something like
>> make-array?
> 
> Well that's the point: they come from something *like* make-arry,
> but not make-array itself. Specifically, in emulating the behaviour
> of the SVDPACK library, the array data comes in from a specific file
> format. I'll store that in some tree-structure and implement a
> version of array-ref which knows how to walk that structure,
> returning 0 for empty entries.

In the end, array-ref and array-set! are all that actually access the
underlying data structure. In my implementations, they reduce to
vector-ref and vector-set!, which can be replaced with appropriate
calls. Aha. I think I begin to see the light now. See below.

> The point is that there should be no explicit dependence in the API
> on a particular implementation strategy. I don't think that any
> exists right now (except possibly using arrays themselves as the
> index objects to arrrays).

I agree about explicit dependence. What I was wondering now is, how
about doing away with the underlying data structure altogether? Or,
more likely, provide a mechanism for attaching the index arithmetic
machinery on any underlying data structure with two operations like
vector-ref and vector-set!. One would then implement make-array so:

  (define (make-array shape o)
     (make-indexed-thing shape
        (make-vector (shape-size shape) o)
        vector-ref
        vector-set!))

And one would implement a tremendously huge but immutable and very
sparse matrix so:

  (define (read-sparse-array shape port)
     (make-indexed-thing shape
        (read-tree port)
        tree-ref
        error))

And one could equally easily use the f64vector and the like that Brad
Lucier wanted.

That should not be hard to do at this point. The index mapping would
remain hidden alright, but the backing vector could be exposed if the
programmer wished to keep a handle on it. That's alright.

I still think the index objects were a separate issue, or else I
misunderstood something about them.
-- 
Jussi