215: Central Log Exchange

by Göran Weinholt

Status

This SRFI is currently in final status. Here is an explanation of each status that a SRFI can hold. To provide input on this SRFI, please send email to srfi-215@nospamsrfi.schemers.org. To subscribe to the list, follow these instructions. You can access previous messages via the mailing list archive.

• Received: 2020-11-02
• Draft #1 published: 2020-11-03
• Draft #2 published: 2020-12-31
• Finalized: 2021-01-11
• Revised to fix errata:
  • 2023-03-31 (Fixed: MSG should be MESSAGE.)
  • 2024-03-13 (Fixed: Pass INFO, not 'INFO, in example.)

Abstract

This SRFI specifies a central log exchange for Scheme that connects log producers with log consumers. It allows multiple logging systems to interoperate and co-exist in the same program. Library code can produce log messages without knowledge of which log system is actually used. Simple applications can easily get logs on standard output, while more advanced applications can send them to a full logging system.

Rationale

Logging is important in many systems, such as large Internet-connected services, where problems are often only evident after the fact. A system designed with logging allows evidence to be gathered and analyzed after an incident. Logs are also used to predict imminent problems and alert operators so that they can take action before an incident occurs.

Scheme lacks a standard logging system. There are some precedents in the Scheme ecosystem, such as Racket's logging system, Andreas Rottmann's spells package and Chibi Scheme's log library. These are examples of high-level logging systems which are intended to be used directly by code that needs to generate log messages. Another precedent is Scheme48's access to the syslog facility, which is a low-level logging system that specifically sends logs to the syslog service.

The logging system in this SRFI is designed to be inserted in between a high-level and a low-level logging system. The main concern is to have a central point that all Scheme code can use to send log messages, and which an application can hook up to whichever log systems it uses. Library code can also choose to use another log system and application developers would then hook up those log systems to the one specified here. By providing a well-known central exchange point through which all log messages are passed, it becomes possible to work generally with log messages throughout the whole application, regardless of which frameworks end up being used to generate and consume log messages.

The API exported by this SRFI consists of one procedure, two parameters and eight constants. The send-log procedure is a convenient procedure for constructing log messages. It can be used in code that just needs to produce log messages and that does not need any advanced log system features. The current-log-fields parameter lets an application set application-wide log fields, such as the syslog facility, thread-local fields such as a subsystem name, or fields that are local to just some smaller context such as a test suite. The current-log-callback parameter is the actual central point for exchanging log messages. Its definition as a parameter object is analogous to the definition of current-output-port (etc.) and allows some advanced features to be built, such as capture of log messages in test runners and custom filtering steps.

This SRFI bases some design elements on the syslog protocol (RFC 5424) and others on the systemd journal. These are both suited as targets for the logs produced by users of this SRFI, but nothing in this SRFI requires the use of them. It is expected that implementers and other users of this SRFI will construct and adapt log messages to be appropriate for the systems they are creating.

Specification

API

Procedure: (send-log severity message [key value] …)

Constructs a log message that contains the arguments and the current log fields. The resulting log message is passed as an argument to the current log callback.

The severity is one of the constants in the table below. Its numeric value is added to the log message with the key SEVERITY.

The optional arguments are grouped into pairs of keys and values and are added to the log message, as in an association list. The value is first converted as described below. The order in which fields appear in the log message is unspecified.

An error is signaled if an odd number of arguments is passed. An error is signaled if one of the keys is not a symbol. Implementations MAY support other types of keys, e.g. keyword arguments, but they MUST be converted to symbols before being added to the log message.

To simplify the handling of values in log callbacks that need to serialize values, every value satisfies at least one of the type predicates listed under Log messages. All other objects are converted to strings as if they had been printed to a string output port by the write procedure.

Implementations MAY convert objects in some other way, but the resulting value in the log message MUST satisfy one of the predicates. A value MAY be converted into an arbitrary number of fields. Implementations MAY append additional fields to the message, such as internal process identifiers, timestamps and other useful context. But it is expected that a high-level logging system will take care of these aspects.

It is an error to mutate the message or one of the values after calling this procedure.

Example:

(send-log INFO (string-append "User " username " logged in")
          'USERNAME username 'REMOTE_IP remote-ip)

Parameter: current-log-fields

This parameter contains a list of additional keys and values that are automatically appended to the fields passed to send-log. The effect is as if that procedure was always invoked as (apply send-log severity message [key value] ... (current-log-fields)).

The default value for this parameter is (), the empty list. Implementations MAY use another default value.

It is an error to mutate any part of the list.

Parameter: current-log-callback

This parameter is bound to the current log callback, which is a procedure that takes a log message as its single argument. Log messages are passed as association lists.

The default value for this parameter is a procedure that is meant to handle log messages that are generated before the application changes the callback. This default procedure buffers an implementation-defined number of messages. When the callback is changed, the new callback receives each message that had been buffered. The buffer is then cleared, to handle the case where the default callback is restored by parameterize.

It is an error to call a log callback with anything except a proper association list that contains the keys SEVERITY and MESSAGE as described below.

Log callbacks MAY discard any field and MAY discard any message. Log callbacks MUST ensure that all messages from the same thread of control arrive in the same order in which they were passed to the callback.

This parameter is meant to be controlled by the application. Libraries MUST NOT change this parameter except as requested by the application.

Log messages

Log messages are association lists that are normally expected to be constructed by send-log, but they can also be passed directly to the log callback with some care. All keys in a log message MUST be symbols and all values MUST satisfy at least one of string?, bytevector?, exact-integer?, error-object? (R7RS) or condition? (R6RS).

Here is a simple example message. The log message ((MESSAGE . "Test message") (SEVERITY . 7)) contains the message "Test message" and its severity is DEBUG.

Applications and library developers are free to design their own log message keys. For the sake of interoperability it is desirable to standardize the names of certain keys, shown in the table below.

^‡ = In the systemd journal this field is confusingly called PRIORITY.
^✻ = In the systemd journal these fields are part of the SYSLOG_IDENTIFIER field, mirroring the obsolete RFC 3164.

An up-to-date list of log message keys will be found in the Scheme Registry. Additions to the list can be made directly in the registry without updating this SRFI.

Parameter implementation

Implementations are free to use any reasonable implementation of parameter objects, such as SRFI 39, but care should be taken that parameterize does not affect the behavior of other threads of control. If an implementation provides R7RS (or a future standard/report with parameters) then it SHOULD use parameters that are compatible with the ones in the base library.

Library name

Systems that implement SRFI 97 should make this library available under the names (srfi :215) and (srfi :215 logging).

Implementation

A sample implementation is provided as an R7RS library. It has no dependencies beyond R7RS Small. A test suite is also provided.

Here is an example of a simple log callback. When a program using this callback runs as a systemd unit, the output will be translated to journal entries marked with severities. A more complete integration with the systemd journal would also pass along the fields given to send-log through the appropriate API.

(current-log-callback
 (lambda (msg)
   (let ((p (current-error-port)))
     (display "<" p)
     (display (cdr (assq 'SEVERITY msg)) p)
     (display ">" p)
     (display (cdr (assq 'MESSAGE msg)) p)
     (newline p))))

(send-log DEBUG "Log callback configured")

Acknowledgements

Thanks to everyone who participated in the discussion on the mailing list.

Copyright

© 2020 Göran Weinholt.

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice (including the next paragraph) shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.