Named quasi-literal constructors


Per Bothner <per@bothner.com>


This SRFI is currently in ``draft'' status. To see an explanation of each status that a SRFI can hold, see here. To provide input on this SRFI, please mail to <srfi minus 108 at srfi dot schemers dot org>. See instructions here to subscribe to the list. You can access previous messages via the archive of the mailing list.


This specifies an extensible reader syntax for named value constructor. A reader prefix is followed by a tag (a symbol), and then expressions and literal text parameters. The tag can be though of as a class name, and the expression and literal text are arguments to an object constructor call. The reader translates the form to a list whose function is $quasi-value$, which is normally bound to a predefined macro.

This propsal is related to SRFI-109 (extended string quasi-literals) and SRFI-107 (XML reader syntax), as they share quite a bit of syntax.


Note the section Discussion: Delimiter options discusses alternative delimiter characters. The syntax examples show two plausible syntax choices: What I call xml-style in red, or scribble-style in green.

Named (quasi-)literals

When adding new datatypes it is useful to add new literals of that type, or at least a compact readable notation for creating instances. SRFI-10 provided one solution. Here is an example, which assumes a URI type for representing encoded Uniform Resource Identifiers (URIs - or generalized URLs):

#,(URI "http://example.com/")

SRFI-10 has a number of problems. One issue is that SRFI-10 conflicts with syntax-case and R6RS. More fundamentallty SRFI-10 resolves the tag name to a constructor function at read type, which requires managing those names using a distinct mechanism. It seems better to use normal scope rules to manage this mapping. The reader is also responsible for calling the constructor, which means the format handled by read is extensible, which is good, but you have to be careful about the security implications, making sure only safe constructor functions are called. Finally, SRFI-10 doesn't integrate with quasi-quotation, and some of us find its syntax a bit ugly.

Instead, this SRFI proposes:


or (if the consensus ends up on Scribble-style syntax):

The Scheme reader translates this to:
($quasi-value$ URI "http://example.com/")

This expression will normally (i.e. if not quoted) invoke the predefined macro $quasi-value$ which expands to a suitable constructor invocation, as specified below.

As related prior art, Caml p4 has an interesting quotation system.

Enclosed unquoted expressions

The arguments to the object initializer aren't always constant. Escaped expressions can be written thus:
(define example-host "example.com")
(Note Scribble-style switches braces and brackets, as discussed later.)
(define example-host "example.com")

Node the use of & for two different purposes: The top-level quasi-literal, and before the unquoted expression. This will be motivated later.

The above example is read as:

($quasi-value$ URI "http://" example-host "/")

Note that enclosed are commonly strings but not always. This example executes an SQL query with a numeric parameter:

#&sql[select * from employees where salary > &{min-salary}]
@sql{select * from employees where salary > @[min-salary]}
Even when the parameter is a string, simple string pasting may be wrong - or dangerous: Consider:
#&sql[select * from employees where name = '&{my-name}']
@sql{select * from employees where name = '@[my-name]'}

Consider what happens if name is constructed by a malicious user and has the value: smith' or ''='. In that case the effective query would be:

#&sql[select * from employees where name = 'smith' or ''='']
@sql{select * from employees where name = 'smith' or ''=''}

This would retrieve all employees. This specification distinguishes literal text from that from an evaluated expression, so an SQL library can do the necessary quoting of special characters.

An example with enclosed expressions:

is translated by the reader to:
($quasi-value$ cname "text1" exp1 "text2" exp2 "text3")

Initial arguments

It is convenient to directly support initial unquoted expressions. Instead of:
#&cname[&{exp1 exp2}text]
@cname{@[exp1 exp2]text}
you can write:
#&cname{exp1 exp2}[text]
@cname[exp1 exp2]{text}

These are almost the same, except that the former is read as

($quasi-value$ cname "" exp1 exp2 "text")
while the latter is read as:
($quasi-value$ cname exp1 exp2 "text")
i.e. without the extra empty string argument. It is most common that both expression will usually evaluate to the same value, but not required.

The initial expression can be especially useful for initial keyword parameters. EXAMPLE NEEDED.

Document processing

One intended application for this extension is document markup. The Scribble system defines a syntax used for writing documents, though it has other uses. It is a kind of template processor with embedded expression in the Racket Scheme dialect. The general Racket syntax for an embedded expression is:
@cmd[datum ...]{text-body}
This SRFI (if using XML-style syntax) switches the roles of {} and [], and uses & instead of @, to be compatible with XML-literals, and also because {} is more commonly used for anti-quotation.

Markup is commonly nested, which suggests that a & in text can be used as an abbreviated extended-datum-literal. Specifically:

is syntatic sugar for:

This nesting of markup justifies why we use the same escape character for both top-level and enclosed forms.

Extra text features

The use of the same escape prefix & as in SRFI-109 (extended string quasi-literals) suggests allowing the same convenience features, including character escapes and indentation handling. See the syntax specification below for details, and see SRFI-109 for examples and motivation.

Resolving to constructor

(This section is mostly Discussion; no definite answers yet.)

The reader creates an $quasi-value$ invocation, but we need to define what this invocation does. We could refrain here from specifying $quasi-value$ except to require that it be in scope when used. However, that seems unfriendly. There should at least be a default definition. Consider

#&cname{pre-exp ...}[abc&{infix-exp1}def&{infix-exp2}...xyz]
@cname[pre-exp ...]{abc@[infix-exp1]def@[infix-exp2]...xyz}
which is read as:
($quasi-value$ cname pre-exp ... "abc" infix-exp1 "def" infix-exp2 ...xyz)
It seems reasonable to have everything in the square backets evaluate to a single string, specifically the same way as if using SRFI-109 quasi-strings:
(cname pre-exp ... #&[abc&{infix-exp1}def&{infix-exp2}...xyz])

How does the implementation of $quasi-value$ value know what forms are in the square brackets? By searching for the first string literal among the macro arguments; that arguments and any remaining are evaluating as a string.

Note a complication that one of the pre-exp or infix-exp arguments might be a literal string. To disambiguate this case, as a special case the reader wraps those literals in a quote form:

#&cname{"A" 123 "B" 456 }[abc&{"K"}xyz]
@cname["A" 123 "B" 456 ]{abc@["K"]xyz}
is read as:
($quasi-value$ cname (quote "A") 123 (quote "B") 456 "abc" (quote "K") "xyz")

This implementation seems OK as a default, but in many cases it will not do the right thing. For example one common Scheme convention is to use make-cname as the name of the procedure for constructing cname objects. You might want to re-arrange the arguments. You might want the enclosed expressions to not be string-ified. So we need a mechanism for programmers to write custom expansions of $quasi-value$. The problem is if you have multiple libraries try to define custom expansions of $quasi-value$. We want these to co-exist without clashing.

A solution is that for a given cname look for the a binding for a special name $quasi-value-transformer$:cname. I.e. the standard building for $quasi-value$ will first look for this binding before doing the above default action.

($quasi-value$ cname form ...)
will map to:
($quasi-value-transformer$:cname form ...)

Using a compound name with a colon should hopefully be portable both to Scheme implementations where colon is a regular identifier constituent, and to Scheme implementations where colon is used for Common Lisp-style prefixed symbols.

This search mechanism may require some implementation-specific hacks, since $quasi-value$ has to first looks for $quasi-value-transformer$:cname and then falls back to a function cname. There may not be a portable way to write this with R6RS syntax-case, though implementations can provide a way to check if there is a lexical binding.

Readtable literals

A possible extension is to support SRFI-10 style read-time literals in certain restricted cases, when all the expressions are literal, and the transformers are available to the reader. This should probably not be the default (for consistency and because of security concerns), but could be supported in an implementation that has programmable read-tables.

Discussion: Delimiter options

There are various choices for delimiter characters in place of &, and in this section we'll discuss some possibilities. It is reasonable to consider both SRFI-108 (this specifiction) and SRFI-109 (extended string quasi-literals)in conjunction with each other. (This discussion refers to the non-terminals defined in the syntax specifications of both SRFI-108 and SRFI-109.)

First let us focus on the escape character in literal-text. After that we will look at characters to use to indicate the start of a top-level extended-string-literal and extended-datum-literal.

Different or same escape characters in literal-text? There are multiple different escape character roles: first we have escapes in string-literal-part. Then in a named-literal-part we have escaped strings and characters (same as in string-literal-part), plus we have nested extended-datum-body. We presumably want to use a single escape character for all the roles in a named-literal-part, so avoid a proliferation of escape characters. Also, for consistency it seems better to use the same escape character and syntax for string and character escapes in both string-literal-part and named-literal-part. The conclusion seems to be we should use the same escape character in all roles (at least within a literal-part). As to which charater to use, the most plausible choices seem to be &, @, or \.

Use & as escape character: Using & is compatible with XML, HTML, SGML, and also "XML literals" embedded in programming languages, including SRFI-107 (XML reader syntax).

Use \ as escape character: Using \ is of course compatible with standard Scheme string literals. Backslash has also been used for as an escape in many languages, for string literals, regular expressions, shells, TeX, and more. If using \ as an escape for SRFI-109 strings, it would be tempting to enhance standard string literals with some of the same features, such as enclosed expressions. However, traditional C-style single-letter escapes, such as \n cause a problem: You either don't allow them in the literal-part of this specification (in which case the latter is not a super-set of standard string escapes), or you need some non-letter prefix character in front of a cname, which is tedious.

Use @ as escape character: Using @ as the escape character goes back to Scribe, TexInfo, and Scribble. These are all markup languages, not programming languages. However, Scribble allows nested Racket Scheme expression, and (if you select the at-exp Racket parser) you can also nest Scribble nested in a top-level Scheme program. If we use @ as the escape character then we might want to switch square and curly braces for better Scribble compatibility.

Braces vs brackets: For extended-string-literal should we use {curly braces} or [square brackets]? I personally think curly seem nicer and perhaps more common, but I might be biased by my experience with JavaFX Script. Furthemore, curly braces are used in SRFI-107 (XML reader syntax), which is already more-or-less implemented in Kawa. On the other hand, Scribble uses square brackets.

Whichever one of brackets or braces is used for unquoted expressions, the other one should be used to enclose a literal-part.

Use braces only: Another option is instead of a single escape character we use some kind of brackets to enclose expressions, as in:

#&[Here is the average: {(/ sum count)}.]
Special characters can be expressed using standard Scheme character or string literals. It is not clear how one would handle a nested extended-datum-body. Special features, like format specifier, and line-paste escapes are also difficult to express.

Use implicit concatenation instead of enclosed expressions: Finally, it is possible to not have any support for expression escapes, but instead have a more compact format for concatenation. For example a string literal right next to an expression, with no space in between, could be defined as concatenation. Thus:

"Here is the average: "(/ sum count)"."
This is pretty fragile (in terms of unintended whitespace for example) though using different start and end string delimiters (for example square brackets) helps:
[Here is the average: ](/ sum count)[.]

Single character to start quasi-literals: Next, when it comes to the the Scheme expression level, we need an unambiguous character or sequence of character to mark the start of a quasi-literal. If we use a single character, it makes sense for that character to match the literal-part escape character, since it easies nested named-literal-part forms. Using & as the start character may cause compatibility problems, since & is a valid <initial> character in standard Scheme, thus it might be difficult to disambiguate from an identifier. However, starting an identifier with & is likely to be rare. Using \ as the start character does not appear to conflict with (draft-)R7RS, but it would be a conflict for many Scheme implementations that use \ as a single-escape character as in Common Lisp. Using @ as the start character does not seem to conflict with standard Scheme, because it is not a valid identifier-start character. However, it might conflict with implementation extensions. (For example Kawa uses @ to name Java-style annotations.)

Starting quasi-literals with # and a dispatch character: Starting quasi-literals with #\ conflicts with character literals. Neither #& or #@ appear problematic.

Recommendation: Either:


Syntax - if using & as escape (XML-style)

expression ::= ...
  | extended-datum-literal
extended-datum-literal ::=
extended-datum-body ::=
  | &cname{expression...}
  | &cname{expression...}[named-literal-part...]
cname ::= identifier
The non-terminal named-literal-part is the same as string-literal-part in SRFI-109 (extended string quasi-literals), except for the support for a nested extended-datum-body.
named-literal-part ::=
    any character except &, [ or ]
  | [named-literal-part...]
  | char-or-entity-ref
  | special-escape
  | &enclosed-part
  | extended-datum-body

The remaining non-terminals match those of SRFI-109 (extended string quasi-literals).

special-escape ::=
  | TBD (at least line pasting and comments)
char-or-entity-ref ::=
  | &#digits;
  | &#xhex-digits;
opt-format-specifier ::= empty
  | ~format-specifier-after-tilde
  | %format-specifier-after-percent
enclosed-part ::=
    &opt-format-specifier{expression ...}
  | &opt-format-specifier(expression...)

Syntax - if using @ as escape (Scribble-style)

This is an alternative syntax, if we decide to use Scribble-style @ escapes.

This is not a true extension/superset of Racket's at-exp mode, but it is the same in most cases. Non-compatible cases include @{foo bar} which Scribble reads as ("foo bar") but I suggest reading as a string "foo bar".

expression ::= ...
  | extended-datum-literal

(In this case, extended-datum-literal and extended-datum-body have the same syntax, so this could be simplified: An @-form is allowed both as an initial character in Scheme forms, and as an escape character in literal parts.)

extended-datum-literal ::=
extended-datum-body ::=
  | @cname[expression...]
  | @cname[expression...]{named-literal-part...}
cname ::= identifier
The non-terminal named-literal-part is the same as string-literal-part in SRFI-109 (extended string quasi-literals), except for the support for a nested extended-datum-body.
named-literal-part ::=
    any character except @, { or }
  | {named-literal-part...}
  | char-or-entity-ref
  | special-escape
  | @enclosed-part
  | extended-datum-body

The remaining non-terminals match those of SRFI-109 (extended string quasi-literals).

special-escape ::=
  | TBD (at least line pasting and comments)
char-or-entity-ref ::=
    @char-or-entity-name; ;;should probably change this
  | @#digits;  ;;should probably change this
  | @#xhex-digits;
opt-format-specifier ::= empty
  | ~format-specifier-after-tilde
  | %format-specifier-after-percent
enclosed-part ::=
    @opt-format-specifier[expression ...]
  | @opt-format-specifier(expression...)


The general form:

#&name{exp1 ... expN}[part1...partM]
@name[exp1 ... expN]{part1...partM}
is translated by the reader to:
($quasi-value$ name exp1 ... exp2 tpart1 ... tpartM)
More precisely:
Tr[#&name{expression...}[content-piece...]]($quasi-value$ name expression... TrContent[content-piece]...)
Tr[#&name[content-piece...]]($quasi-value$ name TrContent[content-piece]...)
TrContent[&name{expression...}[content-piece...]]($quasi-value$ name expression... TrContent[content-piece]...)
TrContent[&name[content-piece...]]($quasi-value$ name TrContent[content-piece]...)
((Rest to be done later, after deciding xml-style vs scribble-style, and based on SRFI-109 (extended string quasi-literals).))


Should provide possible implementation of $quasi-value$, even if not fully portable.

Test suite

Should test translation to reader form. Should test whatever can be tested semi-portably.


Copyright (C) Per Bothner 2012

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.


Author: Per Bothner
Editor: Mike Sperber