This page is part of the web mail archives of SRFI 68 from before July 7th, 2015. The new archives for SRFI 68 contain all messages, not just those from before July 7th, 2015.
On 8/25/05, Michael Sperber <sperber@xxxxxxxxxxxxxxxxxxxxxxxxxxx> wrote: > > I don't understand why it would be a lot of work---that's what the > reference implementation is for. Just drop it in. You mainly have to > adapt the primitive I/O layer, which is usually pretty simple. (You > could even build it easily on a pre-existing ports layer.) You also have to rewrite your existing ports layer to distinguish between the original native ports and the new stream-based ports. There's also still a lot missing from SRFI-68 since it doesn't provide more than some trivial transcoders, and in cases like C's wchar implementation there is no direct way to transcode wstrings - you have to use an external library like iconv. > The bottom two layers are there because they're useful. If all layers > are required, portable code could use whatever layer it wants to. I > don't see why the lower layers (especially the primitive layer) would > necessarily be extremely slow---maybe you can elaborate why you think > this is the case? If you aren't using a systems-level language (or don't want to go through the effort), then you have to base the lower-levels on an existing upper level, which implies all text gets buffered twice. I also have concerns about the stream-layer in general. I really like the idea of being able to treat a port like a list (although given the choice I'd rather work with car and cdr than car+cdr). However, it requires either large amounts of buffering or constant position checks followed by optional seeks. The latter again is extra overhead (and seeks are very expensive and should be avoided whenever possible), so the reference implementation uses the former. But consider the following code which reads an index from the start of a stream in order to determine the offset of a record, seeks to that record and then extracts it (using SRFI-71 let syntax): (define (read-nth-record file n) (let* ((stream1 (open-file-input-stream file)) (index stream2 (read-index stream1)) (offset (nth-record-offset index n)) (stream3 (input-stream-at-position stream2 offset)) (result stream4 (read-record stream3))) (close-input-stream stream4) result)) If the record is the last element of a 10GB file we would need 10GB of memory to do this. A smart compiler and gc could determine the older streams aren't re-used and reclaim them in this case, but what if they were re-used? Or what if they aren't but the compiler can't prove it? Note I had to assume a new procedure input-stream-at-position to seek within the file, which isn't provided by the current SRFI. This is a crucial procedure, but can't efficiently be implemented without closing the current stream and creating a new one from the underlying reader. Likewise we need SET-PORT-POSITION!. Another issue is that buffering doesn't account for mutations performed between reads, so libraries like C provide explicit buffering controls, including non-buffering. The current SRFI effectively prohibits ports from having this level of control. They would in fact be better built on the readers and writers than on streams. -- Alex