275: URIs and IRIs

by Duncan Guthrie

Status

This SRFI is currently in draft status. Here is an explanation of each status that a SRFI can hold. To provide input on this SRFI, please send email to srfi-275@nospamsrfi.schemers.org. To subscribe to the list, follow these instructions. You can access previous messages via the mailing list archive.

Table of contents

Abstract

This SRFI proposes a programming interface for working with RFC 3986 universal resource identifiers (URIs), as well as RFC 3987's generalisation to internationalised resource identifiers (IRIs). This document defines record types, normalization procedures, and conversion between URIs and IRIs. The proposal also specifies a basic programming interface for working with paths in isolation, as this a pervasive usage of URIs and IRIs. Finally, we contribute a test suite to specify the behaviour of normalization with respect to relative references, which has in the past been a source of divergence between implementations.

Issues

none so far

Rationale

RFC 3986 [1] describes an abstract syntax for uniform resource identifiers (URIs) as well as their relative references, which allow documents to be authored without knowing the final publishing location. RFC 3987 [2] defines IRIs, which generalise URIs to Unicode by defining an interpretation of escapes as sequences of UTF-8 octets.

URIs and IRIs are widely used to denote resources across the world wide web, and are the basis of a number of web standards. It is critical to present a programming interface for manipulation of different components in isolation (e.g. paths, hostnames), as working with an URI's coarse string representation directly is error-prone. RFC 3986 defines an abstract syntax and hence conveniently forms the basis of such a programming interface. Further, RFC 3987's generalisation to internationalised identifiers is a natural extension, allowing us to faithfully denote resources in a number of languages, using the universal character set. Indeed, IRIs form the basis of modern standards like Resource Description Format (RDF), a widespread formal model for metadata interchange and knowledge representation. We hence require support for both URIs and IRIs.

Finally, paths are one of the most prevalent applications of URIs and IRIs. We develop paths as an object disjoint from other Scheme types with distinct segment structure, rather than adopting a string representation, because normalisation of paths is a key source of divergence in URI and IRI implementations to date. This object is located in a basic path library. This library structure also lets us expose procedures such as normalisation and merging of base and relative paths directly to the programmer.

Specification

Relative references and polymorphic procedures

RFC 3986 and 3987 distinguish URIs and IRIs, respectively, from their relative references, which must be resolved against a base URI or IRI in order to be used. The main application of relative references is to allow one to refer to resources and author documents without knowing the final publishing location. For example, a graph database may produce RDF documents specified in RDF/XML, where resources are denoted with IRI-relative references, assuming that these documents would be interchanged with another system which mints full IRIs with respect to its hosting location.

Somewhat confusingly, RFC 3986 defines "URI-reference" as the most common usage of URI: either an URI or a relative reference. We follow this common usage by developing polymorphic getters and setters which work on URIs and relative references, with the predicate uri? holding true for both URIs and relative references. This procedure would hence correspond to testing for an RFC 3986 "URI-reference". (RFC 3987 follows an identical convention for IRIs and their relative references, so the same approach is followed for IRIs.)

The most divergent behaviour between widespread implementations of URIs and IRIs has been with respect to normalization of relative references. We explicitly avoid defining path segment normalization for relative references (regardless of whether the path is absolute or relative) because the behaviour is largely undefined by RFC 3986 and 3987 or any current RFC. See the section on normalization for more details and the test suite.

A record type presentation

URIs and IRIs are structured data. Of course, record types are convenient as they generate dedicated getters and setters. More importantly, however, we think that record structures are necessary in this case for normalization procedures to be structure-preserving. Specifically, normalization procedures may alter the path, such that its segments may be mistaken for other parts of the URI or IRI, such as the <://> portion separating scheme from authority, or for the hostname. Updates to authority components or to the path need to similarly ensure that they produce valid URIs and IRIs when serialised.

We argue that a string representation makes it too easy to inadvertently modify the structure, because normalization of individual components may yield a string which, when parsed again, is interpreted differently with respect to the structure. A good example of this is the restrictions on paths given an authority, because a non-empty authority is denoted in an URI using two slashes, which are characters which may also appear in paths.

A string representation of an URI or relative reference, or an IRI or relative reference (procedures uri->string and iri->string) is produced by concatenating string representations of the individual fields (scheme, hostname &c.), with the expected separators between components. For efficiency, no assumption should be made that the contents of individual fields can be checked at this point, which typically would involve additional, redundant parsing.

Pure and impure interfaces

We require implementations to provide a pure-functional interface to URIs and IRIs, but not an in-place interface. The reasoning for this is that if implementors choose data structures optimised to purely functional programming, it is more cumbersome to create an impure interface, whereas it is not as cumbersome for implementors to create a (inefficient) pure-functional interface by copying the URI or IRI before updating in-place.

If an implementation provides disjoint mutable and immutable URI and IRI variants, then it is an error to call the in-place setters on an immutable variant, but the in-place variants should otherwise have equivalent error handling to the pure-functional variants.

Error behaviour of setters

Our design is to provide getters and setters which abstract away the internal representation, working on string representations of URI components. More generally, we suspect that existing implementations largely omit setters because preserving internal consistency is fairly cumbersome on the implementor and programmer, with validity of authority components being defined with respect to the path, and vice versa. The specific challenge is to ensure that setting a given component would not violate the URI grammar, as this may elicit a flat string representation which would have a different structure when parsed again.

First, the scheme, query and fragment components do not depend on validity of other components. Setters invoke the respective parsers on the string representation to be set, and raise an error if the parse failed.

Second, for the path component, validity depends on whether a) any authority component is set; and b) whether we are setting it for an URI, or for a relative reference.

  1. If any authority component is set, then an error is raised if the target path does not begin with a slash, or is otherwise non-empty (see RFC 3986's path-abempty production). An error is raised if there is a parse failure.
  2. If no authority component is set, then we first check whether we are dealing with an URI or a relative reference.
  3. For an URI, if the path has a leading slash (path-absolute production), then the first segment must not be empty (double slash would be parsed as the separator between scheme and authority).
  4. For a relative reference, this is modified so that non-absolute paths must not have an initial segment with a colon in it (as this would be parsed as separator between scheme and authority). An error is raised if a path is to be set which would violate these conditions.

The validity of an authority component likewise depends on the shape of the path. See RFC 3986 ABNF for details.

Normalization

We support three, scheme-independent normalization procedures:

  1. Case normalization, in which U.S. ASCII may be case-insensitive, and in which escaped (percent-encoded) octets have a canonical upper-case form.
  2. Escape normalization, in which percent-encoded octets may be decoded, or characters may be encoded as escapes.
  3. Path segment normalization, in which dotted segments (. and ..) are eliminated.

Case normalization

The scheme and host components of both URIs and IRIs are considered case-insensitive, with other components considered case-sensitive. For URIs, the repertoire of characters is within U.S. ASCII., whereas for IRIs, Unicode characters may appear. Nonetheless, for both URIs and IRIs, only U.S. ASCII is case-insensitive, with characters like "É" never being normalized to a lower-case form like "é".

Escapes (percent-encodings) take the form % 0-F 0-F (hexadecimal digits), and these hexadecimal digits have a canonical upper-case form. For example, %cf would be normalized to %CF. In practice, the sample implementation always parses these into a canonical form as it represents these internally as octets, not in the original string form. If implementations do preserve the original form, they must always support normalization into the canonical upper-case form.

Escape (percent-encoding) normalization

The interpretation of escapes differs for URIs and IRIs.

For URIs, octets may individually be decoded to ASCII characters. Essentially, this occurs if a character is not in the URI reserved range, and if it is permissible within a given URI component (e.g. path). Characters in the reserved range, if encountered in the clear, must never be percent-encoded. Conversely, characters not in the reserved range, and which are not permissible within a given URI component, are encoded as a series of escapes corresponding to a series of UTF-8 octets. This bears particular mention because, while other encodings are valid URIs, the RFC 3986 specification specifically requires this encoding, which enables compatibility with the closely related RFC 3987 specification for IRIs.

IRIs not only generalise the range of characters permissible within the IRI to certain Unicode ranges, but also interpret percent-encodings as UTF-8. Because all URIs are valid IRIs, the normalization of IRIs with respect to percent-encodings is essentially the same as conversion from an URI to an IRI. During conversion, the entire IRI is interpreted as a UTF-8 code sequence, with any percent-encoding not part of a valid UTF-8 sequence being reencoded.

Path segment normalization

Path segment normalization is structurally identical for both URIs and IRIs. It interprets an entire path with respect to two control sequences: . (current working directory) and .. (upper working directory), similar to UNIX paths. Unlike the other two normalization procedures, path segment normalization is undefined for relative references, because the path portion is not meaningful except during relative reference resolution.

It should be noted that path segment normalization is defined for non-relative IRIs with relative paths, such as a URN like <foo:a/b/../.././../../e>. This is a major source of diversion between RFC 3986 implementations. This appears to arise from reusing the remove-dot-segments procedure as defined in RFC 3986 verbatim. In relative reference resolution, this procedure is never called on relative paths, only on absolute paths.

For example, for the (non-relative) IRI <foo:a/b/../.././../../e>, implementations using the RFC 3986 procedure verbatim get <foo:/e>, whereas other implementations get <foo:e>. In the former camp are implementations like the Erlang/OTP system's built-in uri_string [4], and Guile-RDF [6], whereas in the latter camp are implementations like Haskell's network-uri [3] and Chicken's uri-common [10] and uri-generic [11] libraries. We are in the latter camp. A fixed remove-dot-segments, based on Chicken's uri-common library and the SRFI 262 pattern matcher [9], might be implemented as follows:

(define (remove-dot-segments path)
  (let* ([undotted
          (remove-dot-segments/list (path-raw-segments path))]
         [segments
          (list->vector undotted)])
    (cond [(absolute-path? path)
           (make-raw-absolute-path segments)]
          [(and (relative-path? path)
                (fx=? 1 (vector-length segments))
                (u16vector-empty? (vector-ref segments 0)))
           ;; relative path with single empty segment is not meaningful:
           (make-raw-relative-path (vector))]
          [else
           (make-raw-relative-path segments)])))

(define (current-wd? seq) (u16vector= seq (u16vector #x2E)))

(define (parent-wd? seq) (u16vector= seq (u16vector #x2E #x2E)))

(define (remove-dot-segments/list segments)
  (let loop ([ps (vector->list segments)]
             [trailing-slash? #f]
             [lst '()])
    (match ps
      ['()
       (if trailing-slash?
           (reverse (cons (u16vector) lst))
           (reverse lst))]
      [(cons (? current-wd?) rst)
       (loop rst #t lst)]
      [(cons (? parent-wd?) rst)
       (loop rst #t (if (pair? lst) (cdr lst) lst))]
      [(cons x rst)
       (loop rst #f (cons x lst))])))

Additionally, the above suggested implementation of remove-dot-segments is considerably clearer than the stack-based description in RFC 3986, with fewer pattern-matching clauses required.

Basic paths

We additionally specify a basic path sub-library. Paths can be considered vectors of segments which are tagged with whether they have a leading slash (an absolute path). The character set allowed within these paths is equivalent to the characters permissible within an URI or within the range of the universal character set (UCS) permitted within an IRI path segment. This restriction is important because it excludes a number of control characters and characters which can never be typed.

The basic path library specifies the basic operations of path creation from sets of segments (build-path), subscripting (path-ref), functional updates (path-update), and count of segments (path-length). We go a little further and provide utility procedures for path shape (the variants in the RFC 3986 and 3987 ABNF), and for copying paths. Finally, this design allows us to explicitly expose in the programming interface generalist procedures inherited from RFC 3986, such as merge-paths and remove-dot-segments.

Notation and convention

Although we support both URIs and IRIs, for brevity we primarily describe behaviour for IRIs, and omit descriptions of the equivalent procedures for URIs where they behave the same. This works because URIs and IRIs are structurally identical, with the divergence between RFC 3986 and 3987 arising from the generalisation of the character set, and the treatment of normalization.

Type signatures are specified as arrows from input arguments to a single output. Multiple values are denoted (typ ...) and () denotes unit or void (equivalent to R6RS (cond [#f #f])).

In type specifications, when we refer to iri, we refer to both IRIs and IRI-relative references (i.e. the iri type and the relative-iri type). We specify which specific type of IRI with either non-relative-iri or relative-iri. This is a little loose, but best reflects the use of polymorphism in this library interface.

Library references are in the form (srfi :275 <sub-library>) consistent with SRFI 97 [12].

Finally, throughout this document, we enclose IRIs and URIs in angle brackets, e.g. <http://example.org>.

IRI programming interface

Library: (srfi :275 iri)

Syntax: iri
Procedure: (non-relative-iri? ident): any → boolean
Procedure: (iri? ident): any → boolean
Procedure: (iri-scheme ident): non-relative-iri → string?
Procedure: (iri-user ident): iri → string?
Procedure: (iri-host ident): iri → string?
Procedure: (iri-port ident): iri → fixnum?
Procedure: (iri-path ident): iri → path
Procedure: (iri-query ident): iri → string?
Procedure: (iri-fragment ident): iri → string?

IRI record type. The record's fields are derived from the RFC 3986 URI grammar. The procedures for components other than scheme are polymorphic on both IRIs and relative references (for which see below).

Examples:

(define example-IRI (string->iri "http://example.org:80/ex#IRI"))

example-IRI → <http://example.org:80/ex#IRI>

(iri? example-IRI) → #t
(non-relative-iri? example-IRI) → #t

(iri-scheme example-IRI) → "http"
(iri-user example-IRI) → #f
(iri-host example-IRI) → "example.org"
(iri-port example-IRI) → 80
(path-segments (iri-path example-IRI)) → #("ex")
(iri-query example-IRI) → #f
(iri-fragment example-IRI) → "IRI"
  
Procedure: (iri-username+password ident): iri → (string, string)

Helper procedure which splits the user field at the first colon, producing strings corresponding to username and password as two values. This procedure is polymorphic on IRIs and relative references. If there is no colon then the whole user field is returned as first value, and #f as second. If the user field is not set, then #f and #f are returned. A colon which is escaped (percent-encoding %3A) is not considered a delimiter between username and password.

Examples:

(iri-username+password (string->iri"//foo:bar:qux@host"))   → "foo"        "bar:qux"
(iri-username+password (string->iri"//foo%3Abar:qux@host")) → "foo%3Abar"  "qux"
(iri-username+password (string->iri"//@host"))     → ""    #f
(iri-username+password (string->iri "//"))         → #f    #f
(iri-username+password (string->iri "//foo@host")) → "foo" #f
  

We also provide a generic helper procedure for processing strings representing users.

Procedure: (iri-path-string ident): iri → string

In contrast to iri-path, the iri-path-string procedure returns a string representation of the path, which may be the empty string.

Examples:

(define example0 (string->iri "http://example.org:80/ex#IRI"))
(define example1 (string->iri "http://example.org:80#IRI"))
(define example2 (string->iri "//a"))

(path-segments (iri-path example0)) → #("ex")
(iri-path-string example0) → "/ex"

(path-segments (iri-path example1)) → #()
(iri-path-string example1) → ""

(path-segments (iri-path example2)) → #()
(iri-path-string example2) → ""
  
Procedure: (absolute-iri? ident): iri → boolean

Holds true if ident is a non-relative IRI and if fragment is unset. This corresponds exactly to RFC 3987's absolute-IRI production. See also string->absolute-iri.

Examples:

(define example0 (string->iri "http://example.org/ex?cond"))
(define example1 (string->iri "http://example.org/ex#title"))
(define example2 (string->iri "//example.org/ex?cond"))
(define example3 (string->iri "//example.org/ex#title"))

(map absolute-iri? (list example0 example1 example2 example3))
→ (#t #f #f #f)
  
Procedure: (update-iri-scheme ident str): non-relative-iri → string? → non-relative-iri
Procedure: (update-iri-user ident str): iri → string? → iri
Procedure: (update-iri-host ident str): iri → string? → iri
Procedure: (update-iri-port ident str): iri → fixnum? → iri
Procedure: (update-iri-query ident str): iri → string? → iri
Procedure: (update-iri-fragment ident str): iri → string? → iri

Pure-functional setters for IRIs and relative references. Apart from update-iri-port, these procedures take a string, parsing it to the relevant field. It is an error to pass a string which does not conform to the RFC 3987 grammar for that component. The update-iri-scheme procedure is undefined for relative references, and it is an error to call it on a relative reference.

Procedure: (update-iri-path ident path): iri → path | string? → iri

Update an IRI path with either a path object or a string representation. If a string is given, then it must be permissible given the presence of an authority and whether the IRI is a relative reference, otherwise an error is raised with the path as irritant.

If a string is given, then the appropriate string conversion procedure for the path shape (see here), escaping any character not permissible in an IRI path segment. Similarly, if a path is given, any character represented within the path not permissible in an IRI path segment is escaped. Finally, if #f is given, then an empty relative path is set.

Syntax: relative-iri
Procedure: (relative-iri? ident): any → boolean
Procedure: (iri? ident): any → boolean

Relative IRI record type. Relative references do not have a scheme, and it is an error to call iri-scheme on one. The remaining procedures are polymorphic on both relative references and IRIs.

Examples:

(define example-IRI (string->iri "/ex#IRI"))

example-IRI → </ex#IRI>

(iri? example-IRI) → #t
(non-relative-iri? example-IRI) → #f
(relative-iri? example-IRI) → #t

(iri-scheme example-IRI) → <raises an ERROR>
(iri-user example-IRI) → #f
(iri-host example-IRI) → #f
(iri-port example-IRI) → #f
(iri-path example-IRI) → "/ex"
(iri-query example-IRI) → #f
(iri-fragment example-IRI) → "IRI"
  
Procedure: (iri-authority ident): iri | relative-iri → #f | (string? string? fixnum?)
Procedure: (update-iri-authority ident user host port): iri → #f | string? → string? → fixnum? → iri

Authority is derived from the user, host and port. The iri-authority procedure simply retrieves these as multiple values. The update-iri-authority procedure mints a new IRI with these three values set.

Procedure: (iri-equal? A B): iri → iri → boolean

Holds true if the two arguments are IRIs and their fields are equal, or if the two arguments are relative references and their fields are equal. An IRI is never equal to a relative reference, and vice versa. It is an error to call this procedure when either argument is not an IRI or relative reference.

URI programming interface

Library: (srfi :275 uri)

Identical programming interfaces to that of IRIs are given for URIs, with procedures and error messages renaming referencing uri instead of iri. The structure of a URI or URI-relative reference is identical. The programming interface differs in the character set permissible within an URI, i.e. string->uri signals an appropriate error, as do the setters like update-uri-host.

Relative reference resolution

Library: (srfi :275 normalize)

Procedure: (resolve-iri-reference base ref): non-relative-iri → iri → non-relative-iri

Relative reference resolution against a base IRI. An error is signalled if the base IRI is a relative reference. Relative reference resolution is, however, defined for both IRIs and relative references, although it is unusual to resolve a non-relative reference. This procedure is pure-functional as it (usually) involves transforming a relative reference into an IRI.

Simple examples derived from the RDF Turtle test cases (see full test cases at end of document):

(define string-cases (list "g:h" "g" "./g" "g/" "/g" "//g"))

(define base01 (string->iri "http://a/bb/ccc/d;p?q"))
(define base02 (string->iri "http://a/bb/ccc/d/"))
(define base07 (string->iri "file:///a/bb/ccc/d;p?q"))

(define (resolve-with base-iri) ;; Higher-order.  Returns a function.
  (lambda (ref)
    (resolve-reference base-iri (string->iri ref))))

(map (resolve-with base01) string-cases)
→ (list <g:h>
        <http://a/bb/ccc/g> <http://a/bb/ccc/g> <http://a/bb/ccc/g/>
        <http://a/g> <http://g>)

(map (resolve-with base02) string-cases)
→ (list <g:h>
        <http://a/bb/ccc/d/g> <http://a/bb/ccc/d/g> <http://a/bb/ccc/d/g/>
        <http://a/g> <http://g>)

(map (resolve-with base07) string-cases)
→ (list <g:h>
        <file:///a/bb/ccc/g> <file:///a/bb/ccc/g> <file:///a/bb/ccc/g/>
        <file:///g> <file://g>)
  
Procedure: (resolve-uri-reference base ref): non-relative-iri → iri → non-relative-iri

Relative reference resolution against a base URI. Behaviour is structurally identical to that of relative reference resolution for IRIs, as is the error behaviour with respect to base URI.

Normalization

Library: (srfi :275 normalize)

Procedure: (normalize-iri-case ident): iri → iri
Procedure: (normalize-uri-case ident): uri → uri

Normalize case-insensitive components (scheme and host), and convert escaped characters (percent-encodings) to canonical upper-case. These procedures are pure-functional, returning the new identifier. These procedures are structurally identical for both IRIs and URIs, with the only difference being in that they signal an error if the argument is not an IRI or URI respectively. These procedures are also well-defined for IRI and URI-relative references respectively.

Procedure: (normalize-iri-escape ident): iri → iri

Normalize an IRI or relative reference's escapes (percent-encodings), potentially interpreting escapes if part of valid UTF-8 octet sequences as characters. Conversely, characters which are not permissible within an IRI component will be encoded as a series of escapes corresponding to UTF-8 octets. These procedures are idempotent: a fully normalized IRI or relative reference will be normalized to itself, and iri-equal? will hold true between the two. This procedure is pure-functional and structure-preserving.

Procedure: (normalize-uri-escape ident): uri → uri

Normalize an URI or relative reference's escapes (percent-encodings), potentially interpreting escapes as octets in U.S. ASCII. Conversely, characters which are not permissible within an URI component will be encoded as a series of escapes corresponding to UTF-8 octets. This procedure is pure-functional and structure-preserving.

Procedure: (normalize-iri-path-segments ident): iri → iri
Procedure: (normalize-uri-path-segments ident): uri → uri

Normalize path segments depending on the control segments . and ... While path segments of IRI or URI-relative references are not normalized, these procedures simply have no effect and no error is signalled, for parity with the other normalization procedures. While it is relatively uncommon for relative paths to appear in non-relative IRIs or URIs, they are permissible e.g. within URN components. This procedure is pure-functional and structure-preserving.

Procedure: (normalize-iri ident): iri → iri
Procedure: (normalize-uri ident): uri → uri

These procedures are essentially a sequence of the three normalization procedures described previously, albeit the specific order is by escapes, by case and by path segments. These procedures are pure-functional and structure-preserving.

Equivalence and conversion

Library: (srfi :275 utils)

Procedure: (encode-string str): string → string
Procedure: (encode-string str escape-these): string → char-set → string

Process a string, escaping any characters in the provided range. This procedure defaults to encoding no characters (the complement of SRFI 14 char-set:full). Escapes are not interpreted, with the percent-sign treated as a character which is always escaped.

Procedure: (decode-string str): string → string
Procedure: (decode-string str decode-these): string → char-set → string

Process a string, decoding any percent-encoded octets corresponding to characters in the decodeable range. This procedure defaults to decoding no characters (the complement of SRFI 14 char-set:full).

Library: (srfi :275 iri)

Procedure: (string->iri str): string → iri
Procedure: (string->absolute-iri str): string → non-relative-iri

Parse a string as an IRI. An error is raised upon a parse failure, or when encountering a character not permitted within a particular IRI component. Additionally, we go beyond RFC 3987 by raising an error when a percent-encoded escape is part of an invalid sequence of UTF-8 octets. The string->absolute-iri procedure additionally raises an error in absence of a scheme component, and if a fragment component is encountered.

Procedure: (iri->string ident): iri → string

Serialisation of an IRI as a string.

Examples:

(define example-A (string->iri "http://example.org/some/where/place"))
(define example-B (string->iri "urn:/some/where/place"))

(iri->string example-A) → "http://example.org/some/where/place"
(iri->string example-B) → "urn:/some/where/place"
  
Procedure: (get-iri port): port → iri
Procedure: (get-iri port stop?): port → procedure → iri
Procedure: (get-absolute-iri port): port → iri
Procedure: (get-absolute-iri port stop?): port → procedure → iri

The get-iri procedure parses a stream of text from a port as an IRI, stopping if the predicate procedure stop? holds, which is by default eof-object?. Exposing such a low-level procedure is advantageous because it can be used directly in streaming parsers for data which contains IRIs, such as JSON-LD. An error is raised if the port is not a textual input port, and the parse behaviour is identical to string->iri. The get-absolute-iri is related to get-iri in the same way that string->absolute-iri is related to string->iri.

In the sample implementation, this procedure is the basis for string->iri.

Library: (srfi :275 uri)

Procedure: (string->uri str): string → uri
Procedure: (string->absolute-uri str): string → non-relative-uri

Parse a string as an URI. An error is raised upon a parse failure, or when encountering a character not permitted within a particular URI component. Additionally, we go beyond RFC 3986 by raising an error when a percent-encoded escape is part of an invalid sequence of UTF-8 octets. The string->absolute-uri procedure additionally raises an error in absence of a scheme component, and if a fragment component is encountered.

Procedure: (uri->string ident): uri → string

Serialisation of an URI as a string.

Procedure: (get-uri port): port → uri
Procedure: (get-uri port stop?): port → procedure → uri
Procedure: (get-absolute-uri port): port → uri
Procedure: (get-absolute-uri port stop?): port → procedure → uri

The get-uri procedure parses a stream of text from a port as an URI, stopping if the predicate procedure stop? holds, which is by default eof-object?. This low-level procedure is the basis of string->uri and it is provided explicitly so that it can be used in streaming parsers for data which contains URIs. An error is raised if the port is not a textual input port, and the parse behaviour is identical to string->uri. The get-absolute-uri is related to get-uri in the same way that string->absolute-uri is related to string->uri.

Library: (srfi :275 normalize)

Procedure: (iri-eqv? A B): iri → iri → boolean

Two IRIs are equivalent if iri-equal? holds, or, post-normalization with normalize-iri, iri-equal? holds. Similarly, two URIs are equivalent if uri-equal? holds, or, post-normalization with normalize-uri, uri-equal? holds. It is an error to call iri-eqv? when either argument is not an IRI or relative reference. Similarly, it is an error to call uri-eqv? when either argument is not an URI or relative reference.

Procedure: (iri->uri ident): iri → uri

Convert an IRI to an URI. This proceeds by encoding any character within certain ranges (see RFC 3987 ABNF ucschar and iprivate) to a series of escapes corresponding to those octets in UTF-8. This URI is also a valid IRI (albeit not normalized) as all URIs are valid IRIs. This procedure is structure-preserving: (non-relative) IRIs are never transformed into relative references, or vice-versa. It is an error to call this procedure where the argument is not an IRI or relative reference.

Procedure: (uri->iri ident): uri → iri

Convert an URI to an IRI. This procedure can be viewed as upgrading the URI structure to that of an IRI, then normalizing it as an IRI. It is an error to call this procedure where the argument is not an URI or relative reference.

Basic paths

Library: (srfi :275 path)

Syntax: relative-path
Syntax: absolute-path

Procedure: (relative-path? path): any → boolean
Procedure: (absolute-path? path): any → boolean

Two path variants with a common internal segment structure, tagged with whether there is a leading slash.

Procedure: (path-segments path): path → vector(string)
Procedure: (path-segments path decode-these): path → char-set → vector(string)

The common path-segments procedure retrieves the internal segment encoding as a vector of (string) segments. If a character set is provided as an optional argument to path-segments, then percent-encoded escape sequences corresponding to that character set will be decoded.

Procedure: (empty-path? path): any → boolean

Holds true if a path is both relative and has no segments. An absolute path with no segments is not considered empty, as it is represented by the string "/".

Procedure: (rfc-path-rootless? path): any → boolean

Identical to relative-path?

Procedure: (rfc-path-noscheme? path): any → boolean

A variant of relative-path? which only holds true providing the first segment does not contain a colon. This corresponds to the path-noscheme production in the RFC 3986 grammar.

Procedure: (rfc-path-abempty? path): any → boolean

Identical to absolute-path?.

Procedure: (rfc-path-absolute? path): any → boolean

A variant of absolute-path? which only holds true providing the first segment is non-empty (i.e. the path does not start with //). Confusingly, RFC 3986's path-absolute production is a restriction on top of the more general path-abempty.

Procedure: (rfc-path-empty? path): any → boolean

Identical to empty-path?.

Examples for all five above predicates:

(define rfc-abempty-example  (string->path "//some/where/place"))
(define rfc-absolute-example (string->path "/some/where/place"))
(define rfc-rootless-example (string->path "some:thing/where/place"))
(define rfc-noscheme-example (string->path "some/where/place"))
(define relative-empty-example (string->path ""))
(define absolute-empty-example (string->path "/"))

(define all
  (list rfc-abempty-example rfc-absolute-example
        rfc-rootless-example rfc-noscheme-example
        relative-empty-example absolute-empty-example))

(map absolute-path? all)      → (#t #t   #f #f   #f #t)
(map relative-path? all)      → (#f #f   #t #t   #t #f)
(map empty-path? all)         → (#f #f   #f #f   #t #f)

(map rfc-path-abempty? all)   → (#t #t   #f #f   #f #t) ; absolute-path?
(map rfc-path-absolute? all)  → (#f #t   #f #f   #f #f)
(map rfc-path-rootless? all)  → (#f #f   #t #t   #t #f) ; relative-path?
(map rfc-path-noscheme? all)  → (#f #f   #f #t   #t #f)
(map rfc-path-empty? all)     → (#f #f   #f #f   #t #f) ; empty-path?
Procedure: (empty-relative-path): () → relative-path
Procedure: (empty-absolute-path): () → relative-path

Helper procedures which take no arguments and construct a relative path or an absolute path with no segments, i.e. a relative path like "" or an absolute path like "/"

Procedure: (path-length path): path → integer

Retrieve the length of the path segments, a positive integer.

Procedure: (path-ref path k): path → integer → string
Procedure: (path-ref path k decode-these): path → integer → char-set → string

Retrieve the segment at index k as a string. The optional character set argument controls whether to decode any escapes which correspond to a character in the set, and defaults to the complement of SRFI 14 char-set:full (no characters).

Procedure: (path-update path k str): path → integer → string → path
Procedure: (path-update path k str escape-these): path → integer → string → path

Update the path segment at index k with a string. The optional character set argument controls which characters to escape as percent-encoded UTF-8 octets, and defaults to the complement of SRFI 14 char-set:full (no characters).

Procedure: (path=? A B): path → path → boolean

This predicate holds true for two paths A and B, providing that they are both absolute, or are both relative, and that their segments have the same internal structure. An error is raised if either of the arguments are not a path with the argument as irritant.

Procedure: (copy-path): path → path

Copy the given path's segment structure and tag it with whether the input was relative or absolute.

Procedure: (path->string path): path → string
Procedure: (path->string path decode-these): path → char-set → string

Serialise the path as a string. If a character set is provided as an optional argument, then percent-encoded escape sequences corresponding to that character set will be decoded.

Procedure: (path-string? str): string → boolean

This procedure corresponds to the similarly-named Racket procedure, and holds true if its argument is a string which could represent a path. The specific check is that a character is permissible within an URI or is within the universal character set ranges permissible within an IRI (see exported character set char-set:path-allowed).

Procedure: (string->path str): string → path
Procedure: (string->path str escape-these): string → char-set → path

Convert a string to a path. If a character is within the optional character set argument, then it will be escaped. Both relative paths with a colon in the initial segment, and absolute paths with an empty initial segment, are permitted. An error is raised if any percent-encoded escapes are part of sequences which are invalid UTF-8.

Procedure: (string->relative-path str): string → relative-path
Procedure: (string->relative-path str escape-these): string → char-set → relative-path
Procedure: (string->relative-path str escape-these noscheme?): string → char-set → boolean → relative-path

Convert a string to a relative path. If a character is within the optional character set argument, then it will be escaped, and if the additional optional noscheme? argument is provided and holds true, then the path output will be prefixed with ‘./’. If the path has a leading slash, then an error is raised with the input string as irritant.

Procedure: (string->absolute-path str): string → absolute-path
Procedure: (string->absolute-path str escape-these): string → char-set → absolute-path
Procedure: (string->absolute-path str escape-these nonempty-ini?): string → char-set → boolean → absolute-path

Convert a string to an absolute path. If a character is within the optional character set argument, then it will be escaped. If the additional optional nonempty-ini? argument is provided and holds true, and the initial segment is empty (leading double slash), then an error is raised with the input string as irritant.

Procedure: (string->rfc-path-noscheme str): string → relative-path
Procedure: (string->rfc-path-noscheme str escape-these): string → char-set → relative-path

Helper procedure where the final optional argument to string->relative-path is set to #t, then the output path will be prefixed with ‘./’.

Procedure: (string->rfc-absolute-path str): string → relative-path
Procedure: (string->rfc-absolute-path str escape-these): string → char-set → relative-path

Helper procedure where the final optional argument to string->absolute-path is set to #t, meaning that an error is raised for paths which do not correspond to RFC 3986's path-absolute.

Procedure: (vector->relative-path str): vector(string) → relative-path
Procedure: (vector->relative-path str escape-these): vector(string) → char-set → relative-path
Procedure: (vector->relative-path str escape-these noscheme?): vector(string) → char-set → boolean → relative-path

Convert a vector of string segments to a relative path. This procedure behaves similarly to string->relative-path with the exception that that slashes are always escaped, so a slash at the start of the first segment does not raise an error.

Procedure: (vector->absolute-path str): vector(string) → absolute-path
Procedure: (vector->absolute-path str escape-these): vector(string) → char-set → absolute-path
Procedure: (string->absolute-path str escape-these nonempty-ini?): vector(string) → char-set → boolean → absolute-path

Convert a vector of string segments to an absolute path. This procedure behaves similarly to string->absolute-path with the exception that that slashes are always escaped, so a slash at the start of the first segment does not raise an error.

Procedure: (vector->rfc-path-noscheme str): string → relative-path
Procedure: (vector->rfc-path-noscheme str escape-these): string → char-set → relative-path

Helper procedure where the final optional argument to vector->relative-path is set to #t, then the output path will be prefixed with ‘./’.

Procedure: (vector->rfc-absolute-path str): string → relative-path
Procedure: (vector->rfc-absolute-path str escape-these): string → char-set → relative-path

Helper procedure where the final optional argument to vector->absolute-path is set to #t, meaning that an error is raised for paths which do not correspond to RFC 3986's path-absolute.

Procedure: (build-path): () → path
Procedure: (build-path base . segments): string → string → ... → path
Procedure: (build-path base . segments): path → string → ... → path

Variadic procedure to build a path from a base path and a number of additional string segments. If no arguments are given, then the empty path is returned. This procedure corresponds to the similarly-named Racket procedure.

If the base argument is a string, then the procedure behaves like vector->relative-path. If the base argument is a path, then the procedure concatenates the base path with a relative path corresponding to the remaining segments. This is accomplished using the merge-paths procedure described in RFC 3986 section 5.3.2.

By admitting the base as potentially another path, this allows one to build an absolute path from a list of segments, by choosing (empty-absolute-path) as base.

Procedure: (remove-dot-segments path): path → path

Remove dotted segments (‘.’ and ‘..’) by interpreting them alongside the path segment structure. This procedure corresponds to RFC 3986 section 5.2.4.

Procedure: (merge-paths base ref): path → path → path
Procedure: (merge-paths base ref base-has-authority?): path → path → boolean → path

Merge a base path with another path. This procedure corresponds to RFC 3986 section 5.2.3. The optional base-has-authority? argument is important during relative reference resolution, but when merging two paths independently of any IRI or URI, we assume no authority component and default to #f.

Library: (srfi :275 iri)

Procedure: (iri-path-absolute? ident): iri → boolean
Procedure: (iri-path-relative? ident): iri → boolean
Procedure: (iri-path-empty? ident): iri → boolean
Procedure: (iri-path-rfc-abempty? ident): iri → boolean
Procedure: (iri-path-rfc-absolute? ident): iri → boolean
Procedure: (iri-path-rfc-rootless? ident): iri → boolean
Procedure: (iri-path-rfc-noscheme? ident): iri → boolean
Procedure: (iri-path-rfc-empty? ident): iri → boolean

Helper procedures which call the path library's corresponding procedures for path shape on the IRIs' path. Examples:

(define rfc-abempty-example  (string->iri "////some/where/place"))
(define rfc-absolute-example (string->iri "///some/where/place"))
(define rfc-rootless-example (string->iri "some:thing/where/place"))
(define rfc-noscheme-example (string->iri "some/where/place"))
(define relative-empty-example (string->iri "//"))
(define absolute-empty-example (string->iri "///"))

(define all
  (list rfc-abempty-example rfc-absolute-example
        rfc-rootless-example rfc-noscheme-example
        relative-empty-example absolute-empty-example))

(map iri-path-string all)
→ ("//some/where/place"     "/some/where/place"
    "some:thing/where/place" "some/where/place"
    ""                       "/")

(map iri-path-absolute? all)      → (#t #t   #f #f   #f #t)
(map iri-path-relative? all)      → (#f #f   #t #t   #t #f)
(map iri-path-empty? all)         → (#f #f   #f #f   #t #f)

(map iri-path-rfc-abempty? all)   → (#t #t   #f #f   #f #t) ; iri-path-absolute?
(map iri-path-rfc-absolute? all)  → (#f #t   #f #f   #f #f)
(map iri-path-rfc-rootless? all)  → (#f #f   #t #t   #t #f) ; iri-path-relative?
(map iri-path-rfc-noscheme? all)  → (#f #f   #f #t   #t #f)
(map iri-path-rfc-empty? all)     → (#f #f   #f #f   #t #f) ; iri-path-empty?
  
Procedure: (iri-path-segment-ref ident k): iri → integer → string

Retrieve the segment of the path of IRI ident at index k as a string.

Procedure: (iri-path-segment-update path k str): iri → integer → string → path

Update the path segment of the path of IRI ident at index k with a string. Characters which are not permissible within an IRI path segment will be automatically escaped.

Library: (srfi :275 uri)

Equivalent procedures to the procedures described immediately above for IRIs, are defined for URIs.

Character sets

Note that IRIs and URIs export some of the same identifiers where the set of characters is the same in RFC 3986 and 3987: e.g. char-set:reserved. Character sets are named as they are in RFC 3986 and 3987, e.g. char-set:uri-userinfo.

Library: (srfi :275 iri)

IdentifierDescription
char-set:iri-unreservedRFC 3987 iunreserved set.
char-set:iri-privateRFC 3987 iprivate set, comprising additional ranges permissible in IRI queries.
char-set:gen-delimsRFC 3986 gen-delims set.
char-set:sub-delimsRFC 3986 sub-delims set.
char-set:iri-allowedUnion of the above sets.
char-set:reservedUnion of the RFC 3986 gen-delims and sub-delims sets.
char-set:ucscharRange of characters from the universal chararcter set also permissible in an IRI.
char-set:schemeCharacters permitted within RFC 3986 and 3987 scheme components.
char-set:iri-userinfoCharacters permitted within RFC 3987 userinfo components.
char-set:iri-reg-nameCharacters permitted within RFC 3987 non-IP literals.
char-set:iri-segmentCharacters permitted within RFC 3987 path segments.
char-set:iri-segment/nz-ncCharacters in char-set:iri-segment except colon.
char-set:iri-queryCharacters permitted within RFC 3987 query components.
char-set:iri-fragmentCharacters permitted within RFC 3987 fragment components.

Library: (srfi :275 uri)

IdentifierDescription
char-set:uri-unreservedRFC 3986 unreserved set.
char-set:gen-delimsRFC 3986 gen-delims set.
char-set:sub-delimsRFC 3986 sub-delims set.
char-set:uri-allowedUnion of the above sets.
char-set:reservedUnion of the RFC 3986 gen-delims and sub-delims sets.
char-set:schemeCharacters permitted within RFC 3986 and 3987 scheme components.
char-set:uri-userinfoCharacters permitted within RFC 3986 userinfo components.
char-set:uri-reg-nameCharacters permitted within RFC 3986 non-IP literals.
char-set:uri-segmentCharacters permitted within RFC 3986 path segments.
char-set:uri-segment/nz-ncCharacters in char-set:uri-segment except colon.
char-set:uri-queryCharacters permitted within RFC 3986 query and fragment components.
char-set:uri-fragmentCharacters permitted within RFC 3986 query and fragment components.

Library: (srfi :275 path)

Finally, we export a single set, char-set:path-allowed, which is the union of char-set:uri-allowed and char-set:ucschar described above.

Miscellaneous utilities

Library: (srfi :275 utils)

Note this library additionally exports encode-string and decode-string described above.

Procedure: (utf8->string/raise bv): bytevector → string

R6RS's utf8->string procedure silently substitutes the unicode replacement character (U+FFFD) given an invalid UTF-8 sequence. This procedure raises an error of type &assertion-violation with the offending octet as irritant.

Procedure: (percent-encoding->string pct): fixnum → string

Get a string representation of a percent-encoding in its canonical, upper-case form % 0-F 0-F. An error is raised if the input is not a non-negative fixnum less than 256.

(percent-encoding->string #x40) → "%40"
(percent-encoding->string #xFA) → "%FA"
(percent-encoding->string #x00) → "%00"

(percent-encoding->string -12.0  → ERROR)
(percent-encoding->string #x100) → ERROR)
  
Procedure: (percent-encoding->u8-list pct): fixnum → string

Get the UTF-8 octets corresponding to the canonical string representation of the percent-encoding. An error is raised if the input is not a non-negative fixnum less than 256.

(percent-encoding->u8-list #x40) → (#x25 #x34 #x30)
(percent-encoding->u8-list #xFA) → (#x25 #x46 #x41)
(percent-encoding->u8-list #x00) → (#x25 #x30 #x30)

(map integer->char (percent-encoding->u8-list #xFA)) → (#\% #\F #\A)
  
Procedure: (percent-encoding->reverse-u8-list pct): fixnum → string

Get the UTF-8 octets corresponding to the canonical string representation of the percent-encoding in reverse order. An error is raised if the input is not a non-negative fixnum less than 256.

Procedure: (username+password str): string → (string, string)

Split strings retrieved from an IRI or an URI's user component into username and password components. Again, a colon which is escaped is not considered a delimiter between username and password. Examples:

(username+password (iri-user (string->iri"//foo:bar:qux@host")))   → "foo"        "bar:qux"
(username+password (iri-user (string->iri"//foo%3Abar:qux@host"))) → "foo%3Abar"  "qux"
(username+password (iri-user (string->iri"//@host")))     → ""    #f
(username+password (iri-user (string->iri "//")))         → #f    #f
(username+password (iri-user (string->iri "//foo@host"))) → "foo" #f
  

Test suite

In this section, we describe various test cases and the specific behaviour they evaluate. Because RFC 3986 and 3987 only provide examples for relative reference resolution, it is important to specify the exact behaviour of a correct implementation, especially with respect to normalization. It is expected that these test cases could be the basis of a more comprehensive property-based test suite, e.g. test the entire range of reserved and unreserved characters in a particular URI component.

URI case normalization test cases (normalize-uri-case)

All lower-case preserved
<http://example.org/ex#test><http://example.org/ex#test>
Mixed-case scheme to lower-case
<HttP://example.org/ex#test><http://example.org/ex#test>
Mixed-case user preserved
<http://MySelf@example.org/Examp#test><http://MySelf@example.org/Examp#test>
Mixed-case host to lower-case
<http://Example.ORG/ex#test><http://example.org/ex#test>
Mixed case path preserved
<http://example.org/Examp#test><http://example.org/Examp#test>
Mixed case query preserved
<http://example.org/examp?Qua#test><http://example.org/examp?Qua#test>
Mixed case fragment preserved
<http://example.org/examp#TeSt><http://example.org/examp#TeSt>
User percent-encodings to upper-case
<http://%aA@%AA%AB%AC%AD%AE/some/where/place><http://%AA@%AA%AB%AC%AD%AE/some/where/place>
Host percent-encodings to upper-case
<http://%aa%Ab%AC%aD%AE/some/where/place><http://%AA%AB%AC%AD%AE/some/where/place>
Path percent-encodings to upper-case
<http://myname@example.org/%Fa/%FB/%fC><http://myname@example.org/%FA/%FB/%FC>
Query percent-encodings to upper-case
<http://myname@example.org/%FA/%FB/%FC?%ff><http://myname@example.org/%FA/%FB/%FC?%FF>
Fragment percent-encodings to upper-case
<http://myname@example.org/%FA/%FB/%FC#%ff><http://myname@example.org/%FA/%FB/%FC#%FF>

IRI case normalization test cases (normalize-iri-case)

Host non-U.S. ASCII is case-sensitive
<http://CRÊPES.example.org><http://crÊpes.example.org> No equivalent for scheme as that only contains ASCII even in IRIs

URI escape (percent-encoding) normalization test cases (normalize-uri-escape)

User reserved character not escaped
<http://my!name@example.org/ex#test><http://my!name@example.org/ex#test>
Host reserved character not escaped
<http://myname@!example.org/ex#test><http://myname@!example.org/ex#test>
Path reserved character not escaped
<http://myname@example.org/ex!#test><http://myname@example.org/ex!#test>
Query reserved character not escaped
<http://myname@example.org/ex?!a#test><http://myname@example.org/ex?!a#test>
Fragment reserved character not escaped
<http://myname@example.org/ex?a#!test><http://myname@example.org/ex?a#!test>
User reserved escape not decoded
<http://my%40name@example.org/ex#test><http://my%40name@example.org/ex#test>
Host reserved escape not decoded
<http://myname@ex%40ample.org/ex#test><http://myname@ex%40ample.org/ex#test>
Path reserved escape not decoded
<http://myname@example.org/e%40x#test><http://myname@example.org/e%40x#test>
Query reserved escape not decoded
<http://myname@example.org/ex?a%40#test><http://myname@example.org/ex?a%40#test>
Fragment reserved escape not decoded
<http://myname@example.org/ex?a#t%40est><http://myname@example.org/ex?a#t%40est>
User permissible escape decoded
<http://my%2Ename@example.org/ex?a#test><http://my.name@example.org/ex?a#test>
Host permissible escape decoded
<http://myname@example%2Eorg/ex?a#test><http://myname@example.org/ex?a#test>
Path permissible escape decoded
<http://myname@example.org/misc%2Etxt#test><http://myname@example.org/misc.txt#test>
Query permissible escape decoded
<http://myname@example.org/misc.txt?%2E%2E%2E><http://myname@example.org/misc.txt?...>
Fragment permissible escape decoded
<http://myname@example.org/misc.txt#line%31%30><http://myname@example.org/misc.txt#line10>
User illegal characters are escaped
<http://dosh£@crepes.example.org><http://dosh%C2%A3@crepes.example.org>
Host illegal characters are escaped
<http://crêpes.example.org><http://cr%C3%AApes.example.org>
Path illegal characters are escaped
<http://crepes.example.org/in/Rhône><http://crepes.example.org/in/Rh%C3%B4ne>
Query illegal characters are escaped
<http://crepes.example.org/in/Rennes?Dim.‥Sam.><http://crepes.example.org/in/Rennes?Dim.%E2%80%A5Sam.>
Fragment illegal characters are escaped
<http://crepes.example.org/in/Rennes#L'Étage><http://crepes.example.org/in/Rennes#L'%C3%89tage>

IRI escape normalization test cases (normalize-iri-escape)

User permissible escape decoded
<http://dosh%C2%A3@crepes.example.org><http://dosh£@crepes.example.org>
Host permissible escape decoded
<http://cr%C3%AApes.example.org><http://crêpes.example.org>
Path illegal characters are escaped
<http://crepes.example.org/in/Rh%C3%B4ne><http://crepes.example.org/in/Rhône>
Query illegal characters are escaped
<http://crepes.example.org/in/Rennes?Dim.%E2%80%A5Sam.><http://crepes.example.org/in/Rennes?Dim.‥Sam.>
Fragment illegal characters are escaped
<http://crepes.example.org/in/Rennes#L'%C3%89tage><http://crepes.example.org/in/Rennes#L'Étage>
Wholly escaped wholly decoded
<https://en.wiktionary.org/wiki/%E1%BF%AC%CF%8C%CE%B4%CE%BF%CF%82><https://en.wiktionary.org/wiki/Ῥόδος>
Partially normalized wholly decoded
<https://example.org/music/%C3%89irigh'sCuirOrtDoChuid%C3%89adaigh><https://example.org/music/Éirigh'sCuirOrtDoChuidÉadaigh>
Wholly normalized not reencoded
<https://en.wiktionary.org/wiki/Ῥόδος><https://en.wiktionary.org/wiki/Ῥόδος>
Repeated normalization (idempotence)
<https://en.wiktionary.org/wiki/Ῥόδος><https://en.wiktionary.org/wiki/Ῥόδος> In this test case, normalize-iri-escape should be called multiple times, i.e. (compose normalize-iri-escape normalize-iri-escape)

Path normalization test cases (normalize-uri-path-segments and normalize-iri-path-segments)

Absolute path without dotted segments unchanged
<http://example.org/some/where/place><http://example.org/some/where/place>
No authority absolute path without dotted segments unchanged
<urn:/some/where/place><urn:/some/where/place>
Relative path without dotted segments unchanged
<urn:some/where/place><urn:some/where/place>
Absolute path eliminates single dotted segments
<urn:/some/./where/././place/./><urn:/some/where/place/>
Relative path eliminates single dotted segments
<urn:some/./where/././place/./><urn:some/where/place/>
Absolute path empty segments treated like non-empty
<urn:/some//where//place//><urn:/some//where//place//>
Relative path empty segments treated like non-empty
<urn:some//where//place//><urn:some//where//place//>
Single leading slash not normalized
</></>
Multiple leading slashes not normalized
<//><//>
Absolute path reference not normalized (double-dot)
</a/b/../../c></a/b/../../c>
Absolute path reference not normalized (single-dot)
</a/b/././c></a/b/././c>
Absolute path reference not normalized (mixed dotted)
</a/b/../c/././d></a/b/../c/././d>
Relative path reference not normalized (double-dot)
<a/b/../../c><a/b/../../c>
Relative path reference not normalized (single-dot)
<a/b/././c><a/b/././c>
Relative path reference not normalized (mixed dotted)
<a/b/../c/././d><a/b/../c/././d>
1-segment path reference with leading dot not normalized
<./def><./def>
1-segment path reference with leading dot not normalized (colon)
<./abc:def><./abc:def>
Additional relative path case not normalized
<../../abc/./def><../../abc/./def>
Relative path must not be normalized to an absolute path
<foo:a/b/../.././../../e><foo:e> From Haskell network-uri [3]
Empty segments eliminated like non-empty (1)
<http://example.com////../..><http://example.com//> From Webkit [5]
Empty segments eliminated like non-empty (2)
<http://example.com/foo/bar//../..><http://example.com/foo/> From Webkit [5]
Empty segments eliminated like non-empty (3)
<http://example.com/foo/bar//..><http://example.com/foo/bar/> From Webkit [5]
General test case 1
<http://example/a/b/../../c><http://example/c> From Haskell network-uri [3]
General test case 2
<http://example/a/b/c/../../><http://example/a/> From Haskell network-uri [3]
General test case 3
<http://example/a/b/c/./><http://example/a/b/c/> From Haskell network-uri [3]
General test case 4
<http://example/a/b/c/.././><http://example/a/b/> From Haskell network-uri [3]
General test case 5
<http://example/a/b/c/d/../../../../e><http://example/e> From Haskell network-uri [3]
General test case 6
<http://example/a/b/c/d/../.././../../e><http://example/e> From Haskell network-uri [3]
General test case 7
<http://example/a/b/../.././../../e><http://example/e> From Haskell network-uri [3]

URI to IRI conversion (uri->iri)

Wholly escaped path normalized
<https://en.wiktionary.org/wiki/%E1%BF%AC%CF%8C%CE%B4%CE%BF%CF%82><https://en.wiktionary.org/wiki/Ῥόδος>
Partially escaped path normalized
<https://example.org/ceol/%C3%89irigh'sCuirOrtDoChuid%C3%89adaigh><https://example.org/ceol/Éirigh'sCuirOrtDoChuidÉadaigh>

IRI to URI conversion (iri->uri)

Wholly escaped path normalized
<https://en.wiktionary.org/wiki/Ῥόδος><https://en.wiktionary.org/wiki/%E1%BF%AC%CF%8C%CE%B4%CE%BF%CF%82>
Partially escapable path escaped
<https://example.org/ceol/Éirigh'sCuirOrtDoChuidÉadaigh><https://example.org/ceol/%C3%89irigh'sCuirOrtDoChuid%C3%89adaigh>

Relative reference resolution (resolve-iri-reference and resolve-uri-reference)

The following RDF 1.1 Turtle test cases for IRI resolution must succeed. These tests are based on those of RFC 3986, and are structured as triples in which the subject and predicate components are absolute IRIs (URNs), but where the object component is a relative IRI. A base IRI is declared at the top of the Turtle (.ttl) file, and the corresponding N-Triples (.nt) file lists the resolved triples. In the SRFI 275 sample implementation, we inline these tests, with the objects extracted and resolved against the base IRI explicitly.

  1. IRI-resolution-01.ttlIRI-resolution-01.nt (with base IRI <http://a/bb/ccc/d;p?q>)
  2. IRI-resolution-02.ttlIRI-resolution-02.nt (with base IRI <http://a/bb/ccc/d/>)
  3. IRI-resolution-07.ttlIRI-resolution-07.nt (with base IRI <file:///a/bb/ccc/d;p?q>)
  4. IRI-resolution-08.ttlIRI-resolution-08.nt

Additionally, we require the following abnormal test cases, when resolved against base IRI <http://a/b/c/d;p?q>, to yield the following results:

Over-deep ‘..’ traversal is clamped at root, not an error
<../../../g> → <http://a/g>
<../../../../g> → <http://a/g>
<../../../..> → <http://a/>
<../../../../> → <http://a/>
Dotted segments in absolute paths are removed:
</./g> → <http://a/g>
</../g> → <http://a/g>
These are NOT dot-segments; literal names, not traversal:
<g..> → <http://a/b/c/g..>
<..g> → <http://a/b/c/..g>
Dots do not affect query or fragment:
<g?y/./x> → <http://a/b/c/g?y/./x>
<g?y/../x> → <http://a/b/c/g?y/../x>
<g#s/./x> → <http://a/b/c/g#s/./x>
<g#s/../x> → <http://a/b/c/g#s/../x>

Finally, reference resolution must ensure that a path is never elicited which would be mistaken for the double slash after a scheme's colon. This involves prefixing a dot and slash:

Base IRI <f:/a>:
<.//g> → <f:.///g>
Base IRI <f:/a/>:
<..//g> → <f:.///g>

UTF-8 interpretation

In this section, we detail negative test cases which must be rejected. These go beyond the RFC 3986 and 3987 specifications which permit percent-encoded escapes which may correspond to invalid UTF-8 octet sequences. No encodings other than UTF-8 are supported. The following example cases are for the URI component. The same sequences must be rejected in every component in which percent-encoded escapes may appear (every component except scheme and port).

Encode all four byte lengths in a single call
(encode-string "a béc♂d😎e" (char-set-complement char-set:uri-allowed)) → "a%20b%C3%A9c%E2%99%82d%F0%9F%98%8Ee"
Decode everything except ♂ (U+2642, encoded as %E2%99%82)
(decode-string "a%20b%C3%A9c%E2%99%82d%F0%9F%98%8Ee" (char-set-complement char-set:uri-allowed)) → "a béc%E2%99%82d😎e"
Direct encoding treats percent as a character
(encode-string "foo%20bar" (char-set-difference char-set:full char-set:uri-allowed)) → "foo%2520bar"
Escaped percent round trip during direct decoding
(decode-string "foo%2520bar" char-set:full) → "foo%20bar"

The following strings must be rejected by decode-string (in contrast encode-string does not interpret escapes):

Reject overlong 2-byte encoding of U+0020 (space)
"a%C0%A0b"
Reject overlong 3-byte encoding of U+0020
"a%E0%80%A0b"
Reject overlong 4-byte encoding of U+0020
"a%E0%80%80%A0b"
Reject overlong 3-byte encoding of U+00E9 (é)
"a%E0%80%A9b"
Reject incomplete 2-byte sequence (missing continuation byte)
"a%C0b"
Reject incomplete 3-byte sequence (missing third byte for ♂ U+2642)
"a%E2%99"
Reject incomplete 4-byte sequence (missing fourth byte for 😎 U+1F60E)
"a%F0%9F%98"
Reject non-encoded character where a continuation byte is expected
"a%F0%9F%98x"
Reject UTF-8 continuation byte without a leading byte
"a%A9"

Additionally, we require that these sequences are rejected by the different parsers, as well as setters for IRI and URI components (e.g. update-iri-host). The utf8->string/raise procedure is provided for detecting these sequences in external code.

IP literal processing

We expect the following edge-cases to have the corresponding hostname verbatim:

All-zeroes unspecified address
<http://[::]/> → "[::]"
Loopback
<http://[::1]/> → "[::1]"
Trailing ‘::’
<http://[1::]/"> → "[1::]"
Trailing ‘::’, prefix
<http://[2001:db8::]/> → "[2001:db8::]"
Leading ‘::’
<http://[::2001:db8]/> → "[::2001:db8]"
IPv4-mapped
<http://[::ffff:192.0.2.1]/> → "[::ffff:192.0.2.1]"
IPv4-translated (RFC 6052)
<http://[64:ff9b::192.0.2.1]/> → "[64:ff9b::192.0.2.1]"
Full address with port, path, query and fragment
<http://[2001:db8::1]:8080/path?query#fragment> → "[2001:db8::1]"

Negative test cases which must be rejected:

Reject more than one ‘::’ compression marker
<http://[2001:db8:::1]/>
Reject nine groups (maximum is eight)
<http://[2001:db8:a:b:c:d:e:f:1]/>
Reject seven groups with no ‘::’ (too few)
http://[2001:db8:a:b:c:d:e]/< >
Reject octet group exceeding four hex digits
<http://[2001:db8::fffff]/>
Reject same; leading zeros push group to five digits
<http://[2001:00db8::0001]/>
Reject incomplete IPv4 suffix (three octets, not four)
<http://[2001:db8::192.0.2]/>
Reject two separate ‘::’ markers
<http://[2001:db8:a::b::c]/>
Reject only two groups, no ‘::’
<http://[2001:db8]/>
Reject unclosed bracket
<http://[2001:db8>

Parsing expected segments

schemeuserhostportpathqueryfragment
Empty URI: <>
N/A#f#f#f#f#f#f
Empty authority: <//>
N/A#f""#f#f#f#f
Empty user: <//@>
N/A""#f#f#f#f#f
Empty port: <//:>
N/A#f""#f#f#f#f
Empty query: <?>
N/A#f#f#f#f""#f
Empty fragment: <#>
N/A#f#f#f#f#f""
Path which looks like a hostname: <example.org>
N/A#f#f#f"example.org"#f#f
URN-like: <urn:something>
"urn"#f#f#f"something"#f#f
URN-like, path looks like hostname: <urn:example.org>
"urn"#f#f#f"example.org"#f#f
Path which looks like a URN: <./urn:something>
N/A#f#f#f"./urn:something"#f#f
User with colon segment: <http://a:b@c:29>
"http""a:b""c"29#f#f#f
User-like component appears as path: <http::@c:29>
"http"#f#f#f":@c:29"#f#f
Host-like component appears as user: <http://example.org:b@d/>
"http""example.org:b""d"#f"/"#f#f
Padded port as numeric value: <http://example.org:000080>
"http"#f"example.org"80#f#f#f
Query component with question mark: <http://example.org/abcd?efgh?ijkl>
"http"#f"example.org"#f"/abcd""efgh?ijkl"#f
Fragment component with question mark: <http://example.org/abcd#efgh?ijkl>
"http"#f"example.org"#f"/abcd"#f"efgh?ijkl"
Path where first segment looks like host: <http:///some/where/place>
"http"#f""#f"/some/where/place"#f#f
Scheme with nil host: <foo:>
"foo"#f#f#f#f#f#f
Scheme with path, empty host: <foo:////g>
"foo"#f""#f"//g"#f#f
Scheme with path, nil host: <foo:.///g>
"foo"#f#f#f".///g"#f#f
Scheme with non-empty host: <foo://g>
"foo"#f"g"#f#f#f#f
All components filled out: <http://user@example.org:80/some/where/place?qua#ought>
"http""user""example.org"80"/some/where/place""qua""ought"
All components except user filled out: <http://example.org:80/some/where/place?qua#ought>
"http"#f"example.org"80"/some/where/place""qua""ought"
All components except host filled out: <http://user@:80/some/where/place?qua#ought>
"http""user"#f80"/some/where/place""qua""ought"
All components except port filled out: <http://user@example.org/some/where/place?qua#ought>
"http""user""example.org"#f"/some/where/place""qua""ought"
All components except path filled out: <http://user@example.org:80?qua#ought>
"http""user""example.org"80#f"qua""ought"
All components except query filled out: <http://user@example.org:80/some/where/place#ought>
"http""user""example.org"80"/some/where/place"#f"ought"
All components except fragment filled out: <http://user@example.org:80/some/where/place?qua>
"http""user""example.org"80"/some/where/place""qua"#f
Empty host, nil user/port: <http:///some/where/place?qua#ought>
"http"#f""#f"/some/where/place""qua""ought"
Empty user, nil host/port: <http://@/some/where/place?qua#ought>
"http"""#f#f"/some/where/place""qua""ought"
Empty port implies empty host: <http://:/some/where/place?qua#ought>
"http"#f""#f"/some/where/place""qua""ought"
Relative reference, nil host: <////g>
N/A#f""#f"//g"#f#f
Relative reference, path, nil host: <.///g>
N/A#f#f#f".///g"#f#f
Relative reference, non-empty host: <//g>
N/A#f"g"#f#f#f#f
Path which looks like a query: <./p=q:r>
N/A#f#f#f"./p=q:r"#f#f
Relative reference, all components filled out: <//user@example.org:80/some/where/place?qua#ought>
N/A"user""example.org"80"/some/where/place""qua""ought"
Relative reference, all components except user filled out: <//example.org:80/some/where/place?qua#ought>
N/A#f"example.org"80"/some/where/place""qua""ought"
Relative reference, all components except host filled out: <//user@:80/some/where/place?qua#ought>
N/A"user"#f80"/some/where/place""qua""ought"
Relative reference, all components except port filled out: <//user@example.org/some/where/place?qua#ought>
N/A"user""example.org"#f"/some/where/place""qua""ought"
Relative reference, all components except path filled out: <//user@example.org:80?qua#ought>
N/A"user""example.org"80#f"qua""ought"
Relative reference, all components except query filled out: <//user@example.org:80/some/where/place#ought>
N/A"user""example.org"80"/some/where/place"#f"ought"
Relative reference, all components except fragment filled out: <//user@example.org:80/some/where/place?qua>
N/A"user""example.org"80"/some/where/place""qua"#f
Relative reference empty host, nil user/port: <///some/where/place?qua#ought>
N/A#f""#f"/some/where/place""qua""ought"
Relative reference empty user, nil host/port: <//@/some/where/place?qua#ought>
N/A""#f#f"/some/where/place""qua""ought"
Relative reference empty port implies empty host: <//:/some/where/place?qua#ought>
N/A#f""#f"/some/where/place""qua""ought"

Implementation

The sample implementation is written in portable R6RS, and imports SRFIs from the Chez-SRFI grab-bag.

References

Acknowledgements

Thanks to Ivan Raikov and Peter Bex for suggestions on improvements, especially with respect to rejecting invalid UTF-8 sequences.

© 2026 Duncan Guthrie.

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice (including the next paragraph) shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.


Editor: Arthur A. Gleckler