281: Bytevector Utilities

by Peter McGoron

Status

This SRFI is currently in draft status. Here is an explanation of each status that a SRFI can hold. To provide input on this SRFI, please send email to srfi-281@nospamsrfi.schemers.org. To subscribe to the list, follow these instructions. You can access previous messages via the mailing list archive.

• Received: 2026-08-31
• 60-day deadline: 2026-11-01
• Draft #1 published: 2026-09-02

Table of contents

• Abstract
• Issues
• Rationale
• Specification
  • Default behavior of decoders
  • Conventions
  • Optional and required features
  • Composite library
  • Base library
  • Endianness library
  • Serialization library
  • 8-bit integer libraries
  • Arbitrary precision library
  • Multibyte libraries
  • Floating-point libraries
    • Binary32 library
    • Binary64 library
  • Unicode libraries
    • Error-handling mode macro
    • Encoding procedures
    • Decoding procedures
• Implementation
• Acknowledgements
• Appendix 1: Library summaries
• Copyright

Abstract

This SRFI implements a modified version of the R⁶RS’s (rnrs bytevectors (6)) library, which has procedures for the serialization and deserialization of numbers into bytevectors, and conversion to/from Unicode. It also includes some procedures from SRFI 207 that are relevant to all bytevectors, such as serialization/deserialization into strings, and order predicates.

Issues

1. Should native-endianness be a procedure or a value? The R⁷RS Large usually does not have procedures like this: the flonum library and the fixnum library do not use procedures for similar values, such as implementation limits.
2. Should this also include the serialization of number vectors? Or should that be treated somewhere else?
3. Needs more examples (they can be adapted from the R⁶RS).
4. The sample implementation is mostly complete, but is missing a few corner cases, particularly in the serialization library.
5. I want this SRFI to contain an API comparable to SRFI 207 and SRFI 152 adapted to strings and without reference to ASCII characters. However, some of the procedure signatures in SRFI 207 differ from SRFI 152. In addition, there are procedures on u8vectors in SRFI 160 that are different from procedures in either SRFI:
   • bytestring-pad: can take a char argument, string-pad can take start and end arguments. The returned value of string-pad is exactly len, while bytestring-pad is at least len.
   • bytestring-index: does not take start and end arguments. The similar u8vector-index takes one or more u8vectors.
   • bytestring-trim: does not take start and end arguments like string-trim.
   • bytestring-span and bytestring-break: do not take start and end arguments.
   • bytestring-split does not take start and end argument.

Rationale

This section is non-normative.

The R⁶RS bytevector library was voted into the R⁷RS Large with no modifications. Over time, some issues with adopting this library as is have cropped up:

1. The bytevector-copy! procedure has a different argument order than the procedure in the R⁷RS base library.
2. The procedure bytevector-copy is a subset of the behavior in the R⁷RS.
3. The Unicode encoding/decoding procedures and other procedures do not take start and end arguments.
4. The Unicode decoding procedures will always write replacement characters when invalid data is found, while many R⁷RS implementations will raise an exception when invalid data is found.

This SRFI resolves these issues in the following way:

1. The bytevector-copy! procedure is not defined in this SRFI.
2. The bytevector-copy procedure is specified to be the same one as the R⁷RS procedure.
3. The Unicode encoding/decoding procedures and procedures that convert to and from lists are extended to take start and end arguments in a similar way to the other procedures.
4. The Unicode decoding procedures are required to raise exceptions when invalid data is encountered. An extra argument can control what the decoder does.

This SRFI also includes provisions for compatibility with R⁷RS: i.e. an implementation that may not support integers of the sizes specified here, inexact real numbers, or full Unicode.

The procedures with the same names as R⁷RS are extensions of the standard procedures, and implementations are encouraged to use them in their standard libraries.

Specification

Default behavior of decoders

In this SRFI, the default behavior of Unicode decoding procedures such as utf8->string is to raise an error on invalid input. This is different from the R⁶RS behavior, which is to add replacement characters.

This deliberate deviation from the R⁶RS is motivated by the observation that not only do almost all R⁷RS-only implementations that handle bad UTF-8 consistently raise an exception on invalid input, but some implementations that claim to support the R⁶RS also do (Guile and Capy).

Error-raising behavior is more natural. It is the default behavior of textual ports in the R⁶RS, and raising an error in response to bad input allows a user or the programmer to be alerted to corrupted text, as opposed to figuring it out later when the text is silently converted into replacement characters.

The R⁶RS version of the procedures may be obtained by wrapping the procedures with different arguments. For example,

(define (r6rs:utf8->string bv)
  (utf8->string bv 0 (bytevector-length bv) 'replace))

If one specifically wants the R⁶RS procedures, one can import (rnrs bytevectors).

Conventions

All procedures in this SRFI return newly allocated containers (strings, lists, and bytevectors) except that they may not be newly allocated if those containers are empty.

In this SRFI, it is an error if b8 is not an exact integer in [−128, 255]. When b8 is negative, it is interpreted as its two’s complement representation. For example, when -1 is passed to an argument labeled b8, the byte stored in the bytevector is #b11111111.

It is an error if endianness is not big, little or another implementation-defined value.

It is an error if error-handling-mode is not a valid error handling mode (see Unicode libraries).

If end is supplied for a list argument, then list may be improper, but it is an error if list does not have end cons cells in it. If end is not supplied, then it is an error if list is not a list (i.e. finite). This interpretation follows SRFI 274.

Optional and required features

The libraries (srfi 281), (srfi 281 base), (srfi 281 serialization), and (srfi 281 u8) must be supported.

The integer libraries (srfi 281 s8), (srfi 281 u16), etc. are only exported if the implementation supports exact integer objects of those sizes. The library (srfi 281 int) is only exported if the implementation supports exact integer objects of arbitrary size. The (srfi 281 f32) and (srfi 281 f64) libraries are only exported if the implementation supports inexact real numbers. The (srfi 281 unicode) is only exported if the implementation supports all of the Unicode scalar values as characters.

If a library is available, it must export all identifiers defined in it.

If this library is voted into R⁷RS Large, then all features are supported, (scheme bytevector) must be the composite library, and (scheme bytevector) not have any sublibraries.

Composite library

The library (srfi 281) exports all identifiers defined below that are supported by the implementation. For example, if an implementation does not support endianness, then the base library does not export it.

Base library

The library (srfi 281 base) exports the R⁷RS procedures bytevector?, bytevector-length, and bytevector-copy.

Returns #t if obj is a supported endianness: i.e. the symbols big, little, or some implementation-defined symbol.

Returns the native endianness.

Create a bytevector with k elements. When b8 is supplied, the elements are initialized to the appropriate byte value of b8. If b8 is not supplied, the elements are unspecified.

These procedures compare bytevectors lexicographically. Two bytevectors are equal if they have the same number of elements and equal bytes at all valid indices. If two bytevectors differ in length but are the same up to the length of the shorter bytevector, the shorter bytevector is considered to be less than the longer bytevector.

Fills the elements of bytevector from start to end with the appropriate byte value of b8.

Endianness library

The name of ⟨endianness symbol⟩ must be a symbol describing an endianness. The expression (endianness ⟨endianness symbol⟩) evaluates to the symbol named ⟨endianness symbol⟩. If ⟨endianness symbol⟩ is a symbol not supported by the implementation, an error is raised.

Serialization library

These procedures are adapted from SRFI 207 on account of their general applicability to bytevectors. They now take start and end arguments and are no longer required to signal errors.

Returns a newly allocated string consisting of the bytes of bytevector between start and end as upper-case ASCII hexadecimal digits.

It is an error if string between start and end is not made up of pairs of hexadecimal digits.

Returns a newly allocated bytevector, where each sequential pair of hexadecimal digits in string between start and end is converted into the corresponding byte.

It is an error if digits is not a string of two characters or #f. It is an error if the string contains ASCII alphanumeric characters or the character =. If digits is not supplied or #f, it defaults to "+/".

Converts the bytes of bytevector between start and end into a base64 encoded string. The numbers 62 and 63 are encoded as the first and second character, respectively, in digits.

This description defers to RFC 4648.

It is an error if digits is not a string of two characters or #f. It is an error if digits contains an ASCII alphanumeric character or the character =. If digits is not supplied or #f, it defaults to "+/".

Decodes the base64-encoded string between start and end into a bytevector. The first and second characters in digits are decoded into the numbers 62 and 63, respectively. Characters satisfying char-whitespace? are ignored.

This description defers to RFC 4648.

8-bit integer libraries

The (srfi 281 u8) library exports the R⁷RS procedures bytevector-u8-ref and bytevector-u8-set!.

It is an error if the elements of list between start and end are not in [0, 255].

Copies the bytes of list from start to end to a newly allocated bytevector.

Copies the bytes of bytevector from start to end to a newly allocated list.

It is an error if k is not a valid index of bytevector.

Retrieves the kth value of bytevector interpreted as a two’s complement integer.

(bytevector-s8-ref #u8(255) 0) ⇒ -1

It is an error if n is not an exact integer in the interval [-128,127]. It is an error if k is not a valid index of bytevector.

Stores the two’s complement representation of n in bytevector at k.

(define bv (make-bytevector 1))
(bytevector-s8-set! bv 0 -128)
(bytevector-u8-ref bv 0) ⇒ 128

Arbitrary precision library

It is an error if size is not a positive exact integer. It is an error if the integers in [k, k+size) are not valid indices of bytevector.

Return the unsigned integer with size with endianness from bytevector starting at k.

It is an error if size is not a positive exact integer. It is an error if the integers in [k, k+size) are not valid indices of bytevector. It is an error if n is not in [0, 256^size−1].

Stores n as an unsigned size byte number with endianness into bytevector starting at k.

It is an error if size is not a positive exact integer. It is an error if the integers in [k, k+size) are not valid indices of bytevector.

Return the two’s complement integer with size with endianness from bytevector starting at k.

It is an error if size is not a positive exact integer. It is an error if the integers in [k, k+size) are not valid indices of bytevector. It is an error if n is not in [−256^size−1, 256^size−1−1].

Stores n as a two’s complement size byte number with endianness into bytevector starting at k.

Return a newly allocated list of the unsigned/signed integers of size size encoded with endianness from bytevector between start and end.

It is an error if size is not a positive exact integer. It is an error if the elements of list between start and end are not exact integers in the range [0, 256^size−1].

Return a newly allocated bytevector comprising the elements of list between start and end encoded with endianness endianness with size size.

It is an error if size is not a positive exact integer. It is an error if the elements of list between start and end are not exact integers in the range [−256^size−1/2, 256^size−1/2−1].

Return a newly allocated bytevector comprising the elements of list between start and end encoded with endianness endianness with size size.

Multibyte libraries

There are libraries handling signed twos-complement and unsigned integers of size 16, 32, and 64. The sublibrary names are u16, s16, u32, s32, u64, and s64. The metavariable ⟨n⟩ is bound to one of these names. The metavariable ⟨w⟩ is bound to the size in octets of each type of integer. Explicitly, ⟨w⟩ is 2 for 16 bit integers, ⟨w⟩ is 4 for 32 bit integers, and ⟨w⟩ is 8 for 64 bit integers.

It is an error if k to k + ⟨w⟩ − 1 are not valid indices of bytevector.

Retrieve the ⟨w⟩-byte number from bytevector starting at k, with endianness endianness.

It is an error if k to k + ⟨w⟩ − 1 are not valid indices of bytevector. It is an error if k is not a multiple of ⟨w⟩.

Retrieve the ⟨w⟩-byte number from bytevector starting at k, with the native endianness.

It is an error if k to k + ⟨w⟩ − 1 are not valid indices of bytevector. When ⟨n⟩ is an unsigned type, it is an error if n is not in [0, 2^8×⟨w⟩−1]. When ⟨n⟩ is a signed type, it is an error if n is not in [−2^{8×⟨w⟩−1}, 2^{8×⟨w⟩−1}−1].

Store the ⟨w⟩-byte number n into bytevector starting at k, with endianness endianness.

It is an error if k to k + ⟨w⟩ − 1 are not valid indices of bytevector. It is an error if k is not a multiple of ⟨w⟩. When ⟨n⟩ is an unsigned type, it is an error if n is not in [0, 2^8×⟨w⟩−1]. When ⟨n⟩ is a signed type, it is an error if n is not in [−2^{8×⟨w⟩−1}, 2^{8×⟨w⟩−1}−1].

Store the ⟨w⟩-byte number n into bytevector starting at k, with the native endianness.

Floating-point libraries

These procedures convert real numbers to and from IEEE 754 binary floating point representations. These libraries are only exported when inexact real numbers are supported. There is no requirement that any IEEE 754 binary representation is supported as a representation for Scheme numbers.

This SRFI uses the names binary32 and binary64 to refer to the representations. This SRFI also exports identifiers using the R⁶RS names ieee-single and ieee-double: these are deprecated.

Binary32 library

It is an error if the integers [k, k + 3] are not valid indices of the bytevector.

Return an inexact real number that best approximates the value of the binary32 number stored in bytevector starting at k stored in endianness.

It is an error if the integers [k, k + 3] are not valid indices of the bytevector.

Store the closest binary32 approximation of x into bytevector, starting at k with endianness endianness.

It is an error if the integers [k, k + 3] are not valid indices of the bytevector. It is an error if k is not a multiple of 4.

Return an inexact real number that best approximates the value of the binary32 number stored in bytevector starting at k stored in the native endianness.

It is an error if the integers [k, k + 3] are not valid indices of the bytevector. It is an error if k is not a multiple of 4.

Store the closest binary32 approximation of x into bytevector, starting at k with the native endianness.

Binary64 library

It is an error if the integers [k, k + 7] are not valid indices of the bytevector.

Return an inexact real number that best approximates the value of the binary64 number stored in bytevector starting at k stored in endianness.

It is an error if the integers [k, k + 7] are not valid indices of the bytevector.

Store the closest binary64 approximation of x into bytevector, starting at k with endianness endianness.

It is an error if the integers [k, k + 7] are not valid indices of the bytevector. It is an error if k is not a multiple of 8.

Return an inexact real number that best approximates the value of the binary64 number stored in bytevector starting at k stored in the native endianness.

It is an error if the integers [k, k + 7] are not valid indices of the bytevector. It is an error if k is not a multiple of 8.

Store the closest binary64 approximation of x into bytevector, starting at k with the native endianness.

Unicode libraries

Encoded Unicode text has certain features such as overlong encodings and lone surrogates that pose a security issue for programs. This library adds stricter behavior for encoding and decoding procedures.

Each decoding procedure takes an optional argument, error-handling-mode. Standard values for it are the symbols

raise

Raise an exception that satisfies i/o-decoding-error? when the procedure attempts to decode invalid data.

replace

Replace invalid data with one or more U+FFFD characters (replacement characters). Implementations should add replacement characters according to the “U+FFFD Substitution of Maximal Subparts,” Unicode 17.0.0, 3.9.6.

ignore

Ignore invalid data. This is included for compatibility with the R⁶RS: there can be security issues that present themselves when using this mode.

The default value for error-handling-mode is raise.

If an implementation attempts to encode a non-Unicode scalar value, for example a lone surrogate or a character whose integer value is above #x10FFFF, an error is signalled.

Return #t if obj is a valid error-handling mode, and #f otherwise.

Return #t if obj is an exception that was raised by a decoding procedure.

Error-handling mode macro

This library is optional because this syntactic form cannot be implemented in syntax-rules.

The name of ⟨symbol⟩ must be a symbol describing an error-handling mode. The expression (error-handling-mode ⟨symbol⟩) evaluates to the symbol named ⟨symbol⟩. If ⟨symbol⟩ is a symbol not supported by the implementation, an error is raised.

Encoding procedures

Decode the bytes of bytevector between start and end into a newly allocated string, handling invalid data with error-handling-mode.

Returns a newly allocated (unless empty) string whose character sequence is encoded by the given bytevector with bytes between start and end. Errors are handled according to error-handling-mode. The bytevector is decoded according to the appropriate encoding. Endianness is decided in the following way:

If endianness-mandatory? is absent or #f, the procedure determines the endianness according to a byte-order mark (BOM) at the beginning of bytevector if a BOM is present; in this case, the BOM is not decoded as a character. Also in this case, if no BOM is present, endianness specifies the endianness of the encoding. If endianness-mandatory? is a true value, endianness specifies the endianness of the encoding, and any BOM in the encoding is decoded as a regular character.

Decoding procedures

Returns a newly allocated (unless empty) bytevector that contains the UTF-8 encoding of the characters in string from start to end.

Returns a newly allocated (unless empty) bytevector that contains the appropriate encoding of the characters in string from start to end in the given endianness. No byte-order mark is prepended to the bytevector.

If endianness is not specified, it defaults to big.

Implementation

An R⁷RS implementation of this SRFI is available in the SRFI repository. It only requires fixnums of at least 24 bits (SRFI 143), which are very common. Implementers are strongly encouraged to use an optimized implementation, or to implement these procedures as primitives.

The implementation tries to make use of optimized procedures from other SRFIs, such as SRFI 160, when they are available. The implementation will also re-use implementations in (rnrs bytevectors) if available.

On R⁶RS implementations (those that already have the (rnrs bytevectors) library), MIT Scheme, Chibi, CHICKEN (all implementations with explicit-renaming transformers), and Gambit (define-macro), the endianness macro is implemented. Otherwise, it is not supported.

A comprehensive test suite is available that depends on SRFIs 1, 64, 158, and 252. This uses all optional features (and eval), and in full form has millions of assertions. One may wish to comment out parts of the test suite when using it for automated testing.

The sample implementation has been tested on Chibi (bleeding edge from git) and Gauche 0.9.15.

Acknowledgements

Daphne Preston-Kendal discovered and fixed some of the issues with integrating (rnrs bytevectors (6)) into R⁷RS Large.

The R⁶RS editors did most of the work specifying the library.

Daphne Preston-Kendal, John Cowan, and Wolfgang Corcoran-Mathe worked on SRFI 207, which has had some of its procedures adapted to this SRFI.

Appendix 1: Library summaries

Entries with a superscript 7 are procedures reexported from (scheme base). Entries with a superscript 6 are supersets of the semantics of an R⁶RS procedure, usually with the same name.

(srfi 281 base)

endianness? native-endianness⁶ bytevector?^6,7 make-bytevector⁶ bytevector-fill! bytevector-length^6,7 bytevector=? bytevector-copy⁷

(srfi 281 endianness)

endianness⁶

(srfi 281 serialization)

bytevector->hex-string hex-string->bytevector> bytevector->base64 base64->bytevector

(srfi 281 u8)

bytevector-u8-ref^6,7 bytevector-u8-set!^6,7 u8-list->bytevector⁶ bytevector->u8-list⁶

(srfi 281 s8)

bytevector-s8-ref⁶ bytevector-s8-set!⁶

(srfi 281 int)

bytevector-uint-ref⁶ bytevector-uint-set!⁶ bytevector-sint-ref⁶ bytevector-sint-set!⁶ bytevector->uint-list⁶ bytevector->sint-list⁶ uint-list->bytevector⁶ sint-list->bytevector⁶

(srfi 281 u16)

bytevector-u16-ref⁶ bytevector-u16-native-ref⁶ bytevector-u16-set!⁶ bytevector-u16-native-set!⁶

(srfi 281 s16)

bytevector-s16-ref⁶ bytevector-s16-native-ref⁶ bytevector-s16-set!⁶ bytevector-s16-native-set!⁶

(srfi 281 u32)

bytevector-u32-ref⁶ bytevector-u32-native-ref⁶ bytevector-u32-set!⁶ bytevector-u32-native-set!⁶

(srfi 281 s32)

bytevector-s32-ref⁶ bytevector-s32-native-ref⁶ bytevector-s32-set!⁶ bytevector-s32-native-set!⁶

(srfi 281 u64)

bytevector-u64-ref⁶ bytevector-u64-native-ref⁶ bytevector-u64-set!⁶ bytevector-u64-native-set!⁶

(srfi 281 s64)

bytevector-s64-ref⁶ bytevector-s64-native-ref⁶ bytevector-s64-set!⁶ bytevector-s64-native-set!⁶

(srfi 281 f32)

bytevector-binary32-ref⁶ bytevector-ieee-single-ref⁶ bytevector-binary32-native-ref⁶ bytevector-ieee-single-native-ref⁶ bytevector-binary32-set!⁶ bytevector-ieee-single-set!⁶ bytevector-binary32-native-set!⁶ bytevector-ieee-single-native-set!⁶

(srfi 281 f64)

bytevector-binary64-ref⁶ bytevector-ieee-double-ref⁶ bytevector-binary64-native-ref⁶ bytevector-ieee-double-native-ref⁶ bytevector-binary64-set!⁶ bytevector-ieee-double-set!⁶ bytevector-binary64-native-set!⁶ bytevector-ieee-double-native-set!⁶

(srfi 281 error-handling-mode)

error-handling-mode⁶

(srfi 281 unicode)

error-handling-mode? i/o-decoding-error?⁶ utf8->string utf16->string utf32->string string->utf8 string->utf16 string->utf32

Copyright

© 2026 Peter McGoron.

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the “Software”), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice (including the next paragraph) shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED “AS IS,” WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

This document incorporates text from SRFI 207.

© 2020 Daphne Preston-Kendal, John Cowan, and Wolfgang Corcoran-Mathe.

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the “Software”), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice (including the next paragraph) shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED “AS IS,” WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

This document incorporates text from the R⁶RS, which has a simple permissive license.

Michael Sperber, R. Kent Dybvig, Matthew Flatt, and Anton van Straaten (editors), 2007.

We intend this report to belong to the entire Scheme community, and so we grant permission to copy it in whole or in part without fee. In particular, we encourage implementors of Scheme to use this report as a starting point for manuals and other documentation, modifying it as necessary.