211: Scheme Macro Libraries

by Marc Nieper-Wißkirchen

Status

This SRFI is currently in final status. Here is an explanation of each status that a SRFI can hold. To provide input on this SRFI, please send email to srfi-211@nospamsrfi.schemers.org. To subscribe to the list, follow these instructions. You can access previous messages via the mailing list archive.

Abstract

This SRFI describes common syntactic extensions of the syntax-rules macro facility of R5RS and the base R6RS and R7RS libraries. In particular, library namespaces are defined where these extensions can be located and which can be tested against in cond-expand forms.

Note

This SRFI was in draft state for a very long time. The main reason for that was that the author of this SRFI was torn between getting it finalized or withdrawing it. The main reasons that spoke for withdrawal are summarized below. In spite of this, the author finally decided to ask the SRFI editor for finalization because members of the Scheme community expressed their opinion that this SRFI would serve the community more in finalized instead of withdrawn status.

Rationale

Various Scheme implementations define extensions to the syntax-rules macro facility that can be found in R7RS, for example explicit-renaming macros, syntax-case, identifier macros or variable transformers. While different implementation of the same extension will usually be compatible, they cannot usually be found under the same namespace.

This SRFI therefore defines a number of standard libraries under which supporting implementations export the identifiers of those syntax-rules extensions they implement so that portable code can be written without explicitly loading implementation-specific libraries.

Example

Let us assume that we want to write a library that exports the macro skip. The arguments of the skip macro are a non-negative exact integer n and a number of body forms. A skip macro is evaluated by dropping the first n body forms and evaluating the rest as a body.

Writing such a macro is not possible with syntax-rules, but it is possible both with er-macro-transformer and syntax-case. With the following R7RS code, the library works on all implementations that support at least er-macro-transformer or syntax-case, and this SRFI, of course.

    (define-library (skip)
      (export skip)
      (import (scheme base))
      (cond-expand
        ((library (srfi 211 syntax-case))
         (import (srfi 211 syntax-case))
         (begin
           (define-syntax skip
             (lambda (x)
               (syntax-case x ()
                 ((_ n b ...)
                  (with-syntax (((b ...)
                                 (list-tail (syntax->datum #'n) #'(b ...))))
                    #'(letrec* () b ...))))))))
        ((library (srfi 211 er-macro-transformer))
         (import (srfi 211 er-macro-transformer))
         (begin
           (define-syntax skip
             (er-macro-transformer
              (lambda (e r c)
                `(,(r 'letrec*) () ,@(list-tail (cddr e) (cadr e))))))))))

Specification

As a baseline for this specification, we build on the definitions used in the R6RS, especially those described in chapter 12 of the R6RS Standard Libraries.

Syntax objects

Syntax objects, wrapped syntax objects, and (syntax objects representing) identifiers are defined as in section 12.2 of the R6RS Standard Libraries.

Presyntax objects

A presyntax object is a generalization of a syntax object that allows symbols at its leaves. More precisely, a presyntax object is

A preidentifier is a value that is either an identifier or a symbol.

Transformers

A transformer is a transformation procedure or a variable transformer as in section 12.3 of the R6RS Standard Libraries, or an explicit-renaming transformer, an implicitly renaming transformer, a syntactic closures transformer, or a Lisp transformer.

Injecting and renaming

In the context of a transformer being applied to transcribe a macro use of a keyword, injecting a symbol results in an identifier with the same name that behaves as if it were introduced into the code where the keyword was used.

In the same context, renaming a symbol results in an identifier with the same name that behaves as if it were introduced into the code where the keyword was defined.

In the same context, the injection of a presyntax object is a syntax object recursively built from it by replacing each symbol at the leaves with its injection, and the renaming of presyntax object is a syntax object recursively build from it by replacing each symbol at the leaves with its renaming. Both the injection and the renaming preserve any shared or cyclic structure.

Procedures

Identifier predicates

(identifier? obj)

This procedure is defined in section 12.5 of the R6RS Standard Libraries.

(preidentifier? obj)

The preidentifier? procedure returns #t if obj is either an identifier or a symbol, and #f otherwise.

(bound-identifier=? id1 id2)

This procedure is defined in section 12.5 of the R6RS Standard Libraries.

(free-identifier=? id1 id2)

This procedure is defined in section 12.5 of the R6RS Standard Libraries.

Syntax-object and datum conversions

(syntax->datum obj)

This procedure is defined in section 12.6 of the R6RS Standard Libraries.

(presyntax->datum obj)

It is an error if obj is not a presyntax object.

The procedure presyntax->datum strips all syntactic information from the presyntax object obj and returns the corresponding Scheme datum. The difference from syntax->datum is that symbols at the leaves are explicitly allowed and are converted to themselves.

(identifier->symbol id)

It is an error if id is not an identifier.

The procedure identifier->symbol returns the symbolic name of the identifier id, i.e. the result of applying syntax->datum to id.

(preidentifier->symbol id)

It is an error if id is not a preidentifier.

The procedure preidentifier->symbol returns the symbolic name of the identifier id. The difference from syntax->datum is that symbols are allowed and are converted to their own symbolic names.

(unwrap-syntax obj)

It is an error if obj is not a syntax object.

If obj is an identifier or an unwrapped syntax object, the procedure unwrap-syntax returns obj unchanged. If obj is a wrapped syntax object, unwrap-syntax returns the underlying unwrapped syntax object.

(unwrap-presyntax obj)

It is an error if obj is not a presyntax object.

If obj is a preidentifier or an unwrapped presyntax object, the procedure unwrap-syntax returns obj unchanged. If obj is a wrapped syntax object, unwrap-syntax returns the underlying unwrapped syntax object.

(datum->syntax id datum)

This procedure is defined in section 12.6 of the R6RS Standard Libraries.

(construct-identifier id symbol)

It is an error if id is not an identifier or if symbol is not a symbol.

This procedure returns the result of applying datum->syntax to the arguments id and symbol.

Generating temporaries

(generate-temporaries list)

This procedure is defined in section 12.7 of the R6RS Standard Libraries.

(generate-identifier)

(generate-identifier symbol)

The generate-identifier procedure returns a new, unique identifier. The symbolic name of the identifier is symbol if provided, and unspecified otherwise.

Syntax violations

(syntax-violation who message form)

(syntax-violation who message form subform)

This procedure is defined in section 12.9 of the R6RS Standard Libraries.

Note: An implementation that does not have condition objects as defined in chapter 7 of the R6RS Standard Libraries should raise an exception with an appropriate native condition object instead.

Transformers

(make-variable-transformer transformer)

The make-variable-transformer procedure behaves as defined in section 12.3 of the R6RS Standard Libraries, except that transformer can be a transformation procedure, an explicit-renaming transformer, an implicitly renaming transformer, a syntactic closures transformer, or a Lisp transformer.

(er-macro-transformer proc)

It is an error if proc is not a procedure accepting a single argument.

The er-macro-transformer procedure returns an explicit-renaming transformer such that each use of a macro whose keyword is associated with it is transcribed in the following way:

At first, the input form is recursively fully unwrapped, preserving any shared or cyclic structure. Then proc is called with three arguments, the fully unwrapped input form and two procedures rename and compare. It is an error if it does not return a presyntax object. The symbols at the leaves of the presyntax object returned are then replaced by their injections to yield the output form as a syntax object.

The rename procedure accepts a single argument expr. It is an error if expr is not a presyntax object. Rename returns a syntax object that is built from expr by replacing the symbols at the leaves by their renamings. This syntax object may share structure with expr.

The compare procedure accepts exactly two arguments id1 and id2. It is an error if the ids are not preidentifiers. Compare returns the result of free-identifier=? applied to the injections of id1 and id2.

Note: In the preceding paragraphs, er-macro-transformer is defined through an (informally given) operational semantics. This does not mean that er-macro-transformer has to be implemented by following each step of the operational semantics. Any implementation is correct as long as it behaves operationally equivalently. This is like syntax-case is defined in the R6RS through Kent Dybvig's, Robert Hieb's, and Carl Bruggeman's marks and substitutions algorithm but does not preclude other algorithms like the one implemented by André van Tonder in Larceny.

(ir-macro-transformer proc)

It is an error if proc is not a procedure accepting a single argument.

The ir-macro-transformer procedure returns an implicitly renaming transformer such that each use of a macro whose keyword is associated with it is transcribed in the following way:

At first, the input form is recursively fully unwrapped, preserving any shared or cyclic structure. Then proc is called with three arguments, the fully unwrapped input form and two procedures inject and compare. It is an error if it does not return a presyntax object. The symbols at the leaves of the presyntax object returned are then replaced by their renamings to yield the output form as a syntax object.

The inject procedure accepts a single argument expr. It is an error if expr is not a presyntax object. Inject returns a syntax object that is built from expr by replacing the symbols at the leaves by their injections. This syntax object may share structure with expr.

The compare procedure accepts exactly two arguments id1 and id2. It is an error if the ids are not preidentifiers. Compare returns the result of free-identifier=? applied to the renamings of id1 and id2.

(lisp-transformer proc)

It is an error if proc is not a procedure accepting a single argument.

The lisp-transformer procedure returns a Lisp transformer such that each use of a macro whose keyword is associated with it replaces the macro use with the injection of the datum resulting from invoking proc on the datum representing the macro use.

Syntactic closures

The following procedures are mentioned without being given any semantics. The reason is that the usual semantics as given in the MIT/GNU Scheme manual doesn't allow the definition of robust unhygienic macros. (The reasons for that are similar to those why the unmodified explicit-renaming system is unsuitable for unhygienic macros.)

However, while it is clear (as this SRFI shows) how to extend the explicit-renaming facility so that it becomes compatible with syntax-case and allows robust unhygienic macros, a universally accepted solution for syntactic closures hasn't been proposed yet. The author of this SRFI has implemented an extended version of syntactic closures compatible with syntax-case in his Unsyntax Scheme, but this extension is incompatible with the canonical implementation of syntax-rules with syntactic closures, which, in turn, is incompatible with robust unhygienic syntactic closure macros.

Regardless, the names of the procedures forming the syntactic closures facility are given here so that the exports of (srfi 211 syntactic-closures) are fixed. Despite a missing specification here, a Scheme programmer can still refer to (srfi 211 syntactic-closures) to write programs that will be portable to most implementations of syntactic closures in practise.

(sc-macro-transformer proc)

(rsc-macro-transformer proc)

(make-syntactic-closure environment free-names form)

(close-syntax form environment)

(capture-syntactic-environment proc)

(sc-identifier=? obj)

(make-synthetic-identifier identifier)

Syntax

Macros

(keyword datum …)

(keyword datum … . datum)

keyword

(set! keyword datum)

These macro uses are defined in section 9.2 of the R6RS.

Note: Every implementation must support the first form of a macro use and should support all four forms. It must support the third form if it provides identifier-syntax, it must support the third and fourth form if it provides make-variable-transformer, and it must support all four forms if it provides syntax-case.

Custom ellipsis identifiers

(with-ellipsis ellipsis identifier expression or definition …)

Syntax: Ellipsis identifier is an identifier.

Semantics: The expressions and definitions are spliced into the surrounding lexical context just as if they were contained in a begin sequencing construct but expanded in an extended syntactic environment. In this extended syntactic environment, an ellipsis in a macro pattern or template is an identifier such that binding the ellipsis identifier would shadow free references to it.

Note: The normal behavior is that an ellipsis in a macro pattern or template is an identifier that has the same binding as ....

Rationale: In the syntax-case system, the syntax-rules facility is destructured into its natural components, a pattern matcher (syntax-case) and a templating engine (syntax). Accordingly, in such a system, syntax-rules is naturally defined in terms of syntax-case and syntax, and the R6RS gives an according definition of syntax-rules in section 12.8 of its Standard Libraries.

R7RS adopted all of the syntax-rules extensions of SRFI 46: Basic Syntax-rules Extensions, including the possibility of specifying a custom ellipsis. The with-ellipsis form described here gives the same power to syntax-case and syntax and ensures that syntax-rules is still a simple derived form of syntax-case and syntax.

The extended form

    (syntax-rules ellipsis (pattern literal …)
      syntax rule …)

of a syntax-rules transformer is equivalent to

    (with-ellipsis ellipsis
      (syntax-rules (pattern literal …)
        syntax rule …))

The following example comes from the GNU Guile Reference Manual.

    (define-syntax define-quotation-macros
      (lambda (x)
        (syntax-case x ()
          ((_ (macro-name head-symbol) ...)
           #'(begin (define-syntax macro-name
                      (lambda (x)
                        (with-ellipsis :::
                          (syntax-case x ()
                            ((_ x :::)
                             #'(quote (head-symbol x :::)))))))
                    ...)))))
    (define-quotation-macros (quote-a a) (quote-b b) (quote-c c))
    (quote-a 1 2 3)                                                 ⟹ (a 1 2 3)

Parsing input and producing output

(syntax-case expression (literal …) syntax-case clause)

This syntax is defined in section 12.4 of the R6RS Standard Libraries.

Contrary to that specification, it is not a syntax violation if the ellipsis or underscore appears in (literal …). If the ellipsis or underscore appear in (literal …), they are treated as literal identifiers.

Rationale: This extension to R6RS's syntax-case is necessary so that R7RS's syntax-rules can be easily defined in terms of syntax-case.

Note 1: The author of this SRFI believes that the exclusion of the ellipsis and the underscore in the list of literal identifiers was a mistake of the R6RS.

Note 2: In the lexical scope of an with-ellipsis form, the meaning of the ellipsis is changed.

(syntax template)

This syntax is defined in section 12.4 of the R6RS Standard Libraries.

Note: In the lexical scope of an with-ellipsis form, the meaning of the ellipsis is changed.

Derived syntax

(with-syntax ((pattern expression) …) body)

This syntax is defined in section 12.8 of the R6RS Standard Libraries.

Note: In the lexical scope of an with-ellipsis form, the meaning of the ellipsis is changed.

(quasisyntax template)

This syntax is defined in section 12.8 of the R6RS Standard Libraries.

Note: In the lexical scope of an with-ellipsis form, the meaning of the ellipsis is changed.

unsyntax

This auxiliary syntax is defined in section 12.8 of the R6RS Standard Libraries.

unsyntax-splicing

This auxiliary syntax is defined in section 12.8 of the R6RS Standard Libraries.

Transformers

(identifier-syntax template)

(identifier-syntax (id1) template1) ((set! id2 pattern) template1))

This syntax is defined in section 11.19 of the R6RS.

Macro definitions

(define-macro keyword transformer)

Syntax: Keyword is an identifier and transformer an expression.

Semantics: This syntax definition is equivalent to the syntax definition

    (define-syntax keyword
      (lisp-transformer transformer))

(define-macro (keyword . formals) body1 body1 …)

Syntax: Keyword is an identifier, formals a formal arguments list, and transformer an expression.

Semantics: This syntax definition is equivalent to the syntax definition

    (define-syntax keyword
      (lisp-transformer
        (lambda (exp)
          (let-values ((formals (apply values (cdr exp))))
            body1 body2 …))))

Syntax parameters

(define-syntax-parameter keyword expression)

This syntax is defined in SRFI 139.

(syntax-parameterize ((keyword expression) …) body)

This syntax is defined in SRFI 139.

Libraries

An implementation does not have to provide all the libraries below. If it provides a library, it has to provide it as a whole and not just a subset of it.

The composite library

(srfi 211)

Exports all identifiers defined in this SRFI.

The low-level macro facility of the R4RS

(srfi 211 low-level)

Exports the identifiers

Rationale: Although this thirty-year-old low-level macro facility is usually not mentioned when talking about macro systems, it has the advantage that it is on the one hand simple and abstract enough to serve as a common denominator between Schemes that have explicit-renaming macro transformers and Schemes that have syntax-case and on the other hand powerful enough so that it can define syntax-rules as a macro.

In a certain sense, this macro facility is syntax-case without syntax-case.

Compared to the explicit-renaming facility, which is quite comparable to this low-level macro facility, it does not assume that abstract syntax objects are Scheme datums but enforces the use of unwrap-syntax. While this can be no-op when implemented on top of explicit-renaming macro transformers, it is not in the case when implemented on top of the marks and substitutions implementation of syntax-case. Similarly, it does not assume that identifiers that are bound-identifier=? are already eq?. These further abstractions make the system compatible to the usual syntax-case systems.

The system is, in principle, as expressive as the syntax-case system (but without all bells and whistles) and thus more powerful than explicit renaming.

To the explicit-renaming facility, it adds that a macro transformer can also be given by procedure taking a syntax object and returning a syntax object. Thanks to the abstract nature of a syntax object, this can be easily achieved by implementations.

Explicit-renaming macros

(srfi 211 explicit-renaming)

Exports the identifiers

Note 1: In legacy implementations, to test whether binding one identifier would also bind the other, the equivalence predicate eqv? is used in the context of explicit-renaming macros. In general, however, this equivalence predicate is too fine as soon as the system is augmented with a means to capture the syntactic environment of an identifier as the following code shows:

    (let ((x #f))
      (let ((x #f)) ...))

In the context of ... binding one of the two identifiers named x would certainly also bind the other. However, both identifiers cannot be equivalent because the syntactic environment of the second is extended by the binding of the first.

Consequently, macro programmers should use bound-identifier=? instead of eqv? whenever one of the libraries (srfi 211 low-level) or (srfi 211 syntax-case) is provided as well.

Note 2: In legacy implementations, to implement non-hygienic macros, raw symbols are inserted into the macro output in the context of explicit-renaming macros. In general, however, this cannot lead to robust macros in a typical implementation as the following example shows:

    (define-syntax loop
      (er-macro-transformer
        (lambda (x r c)
          (let ((body (cdr x)))
            `(,(r 'call-with-current-continuation)
              (,(r 'lambda) (exit)
               (,(r 'let) ,(r 'f) () ,@body (,(r 'f)))))))))

    (define-syntax foo
      (syntax-rules ()
        ((foo x) (loop (exit x)))))

    (foo 42)

The macro writer may expect that the expression (foo) evaluates to 42. Instead, in most implementations the program will fail because the identifier exit in the template of the foo macro is effectively renamed and does not capture the binding of the identifier exit introduced by the loop macro.

The following rewrite of foo will solve this problem.

    (define-syntax foo
      (er-macro-transformer
        (lambda (x r c)
          `(,(r 'loop) (exit ,(cadr x))))))

This rewrite does not solve another problem, however. The expression

    (let ((exit 42))
      (foo exit))

evaluates to the procedure exit introduced by the loop macro, not to 42, which may have been expected. The real problem lies in the loop macro, which should have renamed exit in the context of the macro invoking it.

Consequently, in legacy implementations, macro programmers should not return raw symbols from their transformers, but use construct-identifier from (srfi 211 low-level) or datum->syntax from (srfi 211 syntax-case) when any of these two libraries is provided as well.

Note 3: In legacy implementations, it may be an error if the rename procedure is applied to a non-symbol. It is an error if the compare procedure is applied to non-preidentifiers.

Note 4: In view of the above, this specification demands that er-macro-transformer is implemented in a way so that raw symbols in the output are closed in the syntactic environment of the invoking macro keyword. Furthermore, it is demanded that the rename procedure accepts arbitrary datums representing Scheme code and that the compare procedure closes raw symbols in the syntactic environment of the invoking macro keyword.

Implicitly renaming macros

(srfi 211 implicit-renaming)

Exports the identifier

Note: Most of the notes to explicit renaming apply mutatis mutandis here as well.

The syntax-case system

(srfi 211 syntax-case)

Exports the identifiers

Note: As syntax-case must accept ... and _ as literal identifiers, syntax-case exported by (srfi 211 syntax-case) cannot be the same as syntax-case exported by (rnrs syntax-case (6)).

Identifier syntax

(srfi 211 identifier-syntax)

Exports the identifier

Note: If this library is present, a macro transformer will be invoked if its associated keyword is referenced as an expression. The more general form of identifier-syntax that permits the transformer to determine what happens when set! is used may not be supported by legacy implementations unless the (srfi 211 variable-transformer) library is present as well.

Variable transformers

(srfi 211 variable-transformer)

Exports the identifier

Custom ellipsis identifiers

(srfi 211 with-ellipsis)

Exports the identifier

Syntax parameters

(srfi 211 syntax-parameter)

Exports the identifiers

Old-style Lisp macros

(srfi 211 define-macro)

Exports the identifiers

Note: A legacy implementation that does not support define-syntax, let-syntax, and letrec-syntax need not define lisp-transformer.

Remark: The syntax and procedures defined by the library (srfi 211 define-macro) should be considered deprecated. Their use is only recommended by code that needs to be compatible with Scheme implementations lacking a working hygienic macro facility and for demonstration or educational purposes. Scheme implementations that do have a working hygienic macro facility are even encouraged not to provide this library by default.

Syntactic-closures macro transformers

(srfi 211 syntactic-closures)

Exports the identifiers

Presyntax objects

(srfi 211 presyntax)

Exports the identifiers

Implementation

The implementation is necessarily distinct for every Scheme implementation but in any case almost trivial. The identifiers of each supported extension of syntax-rules just have to be re-exported in the standardized libraries defined above.

Implementers are highly encouraged to provide an implementation of the low-level macro facility (srfi 211 low-level), which can serve as a foundation for syntax-rules and as a common denominator between the different Scheme implementations.

The Unsyntax implementation of the Scheme programming language provides all of the above libraries. Its implementation of er-macro-transformer follows all of the above recommendations.

Acknowledgements

Thanks to the inventors and implementers of the various extensions to the syntax-rules macro facility that are covered by this SRFI.

The idea for with-ellipsis comes from GNU Guile.

Alex Shinn came up with the final title of this SRFI.

Thanks to John Cowan and Alex Shinn for their insistence to make this SRFI as much as independent from non-stable external sources.

Wording has been taken from the R6RS, the R7RS, and R6RS Syntax-Case Macros.

© 2021 Marc Nieper-Wißkirchen.

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice (including the next paragraph) shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

Editor: Arthur A. Gleckler