This page is part of the web mail archives of SRFI 50 from before July 7th, 2015. The new archives for SRFI 50 contain all messages, not just those from before July 7th, 2015.
> From: bear <bear@xxxxxxxxx> I'll mostly answer your points in order but the last one is the most interesting: > I think that if we have the new procedures char-cased? and > char-uncased? we do not need the proposed char-letter? > predicate. (I argue below that your definition of "cased" characters is problematic but that's not the main point here.) A while back, tb argued that the case-mapping procedures of R5RS could simply be dropped. There's something to that. In fact, R6RS could go further -- it could: DROP: RETAIN: (case and classes) (type, order, integer isomorphism) char? char-ci=? char=? char-ci<? char<? char-ci>? char>? char-ci<=? char<=? char-ci>=? char>=? char-alphabetic? char->integer char-numeric? integer->char char-whitespace? char-upper-case? char-lower-case? char-upcase char-downcase string-ci=? string-ci<? string-ci>? string-ci<=? string-ci>=? ADD: (metacircularity procedures) char-delimiter? string->character string->string string->symbol-name form-identifier Why do that? I'm not convinced we should but the arguments for doing so would include: ~ it would remove from R5RS all traces of the naive approach to character case ~ it would remove from R5RS the culturally biased character class "alphabetic" ~ it would evaded the tricky problem of define "numeric" usefully yet without cultural bias ~ those changes would leave only the class CHAR-WHITESPACE? which seems particularly odd in isolation ~ the ability to write metacircular programs would still be present -- and improved ~ the basic structure of the CHAR? type, a well-ordered set isomorphic to a subset of the integers, would be retained Why not do it? ~ pedagogical reasons -- for the portable character set, the metacircularity procedures can be defined using the dropped procedures ~ practical reason -- it wouldn't leave enough standard machinary in Scheme to parse simple formats like "whitespace separated fields" ~ practical reason -- implementors will want to provide all of the procedures in the DROP column for years to come, at least. Useful libraries will continue to rely on them. It is worthwhile to (continue to) say what they should mean. But, on to the proposed revisions to the proposed revisions to the revised^5 specification: >> It should say: >> Returns the name of symbol as a string. [...] will be in the >> implementation's preferred standard case [...] >> will prefer upper case, others lower case. If the symbol was >> returned by string->symbol, [....] string=? to >> the string that was passed to string->symbol. [....] > I would propose instead: > Returns the name of symbol as a string. [...] all cased > characters in the identifier (see the definition of char-cased? > for a precise definition of cased and uncased characters) will > be in the implementation's preferred standard case [....]. If > the symbol was returned by string->symbol, the case of the > characters in the string returned will be the same as the case > in the string that was passed to string->symbol. [....] > Rationale; I think it's simply clearer. The above wording > specifically permits uncased characters (ie, characters which do not > conform to "normal" expectations of cased characters) to be present > in lowercase in identifiers even if the preferred case is uppercase, > and presumably vice versa. Huh. I thought that my wording permitted that already. I mostly dislike your wording. This part: > all cased characters in the identifier [...] will be in the > implementation's preferred standard case seems too strong to me. I'd be willing to accept it if (a) we nail a good STRING->SYMBOL-NAME definition for the "Unicode Identifiers" draft; (b) prove that the property you named is true for that STRING->SYMBOL-NAME and for all future versions of Unicode. This part: > If the symbol was returned by string->symbol, the case of the > characters in the string returned will be the same as the case > in the string that was passed to string->symbol. is too weak. The two strings must be STRING=?. For example, a Unicode STRING->SYMBOL must not canonicalize its argument (and STRING=? is a codepoint-wise comparison). >> With regard to character class predicates such as char-alphabetic? >> [...] >> The procedure char-alphabetic? is deprecated. New programs should >> usually use char-letter? (see below) instead. char-alphabetic? has a >> precise definition in terms of char-letter?: >> (define (char-alphabetic? c) >> (and (char-letter? c) >> (char-upper-case? (char-upcase c)) >> (char-lower-case? (char-downcase c)))) > This is not how linguists use the term "alphabetic." Please do > not propose "alphabetic" as a procedure to use to mean this, as > it will frustrate and confuse people. It's true that that is not how linguists use the term "alphabetic". It's also true that not all "letters", in the sense of Unicode, are alphabetic characters. For example, ideographic characters are categorized in Unicode as "letters"; syllabaries are classified as "letters". In a Unicode implementation, a linguistic definition of CHAR-ALPHABETIC? would be a subset of letters generally and would include both characters which are not cased (U+13A0 ("CHEROKEE LETTER A")) and characters with no single-character case-mappings (U+00DF ("LATIN SMALL LETTER SHARP S")). That would, in some sense, be a an interesting procedure to have around -- but really it belongs in a general library for linguistic text processing (along with many other procedures). Worse, a linguistically proper definition of CHAR-ALPHABETIC? would be upwards incompatible with R5RS which requires that alphabetic characters have upper and lowercase forms (which are themselves characters). When thinking about how to handle this situation, I reasoned this way: 1) One use for the R5RS character classes is to write programs which process s-expressions (e.g. source text) over the portable character set. This use should be preserved. 2) Another use for the R5RS character classes is to write programs which parse other simple kinds of syntax. For example, parsing a line of text into white-space separated fields. This use should be preserved and expanded. For example, CHAR-LETTER? allows for a field of letters which are not alphabetic characters or which are alphabetic but not case-mapped in the naive way. 3) The R5RS character classes have never been well suited for linguistic processing over anything but the portable character set. Their use for such purposes for extended characters is unrealistic. 4) Upward compatability with R5RS is desirable. 5) The specifications for the character classes defined in R6RS should be consistent with definitions that satisfy the usual expectations of a Unicode programmer. In other words, in a Unicode-based implementation, these procedures should function as a useful subset of a comprehensive library for Unicode text processing. So, I proposed: adding CHAR-LETTER? which is (consistent with being) the generalization of CHAR-ALPHABETIC? to all "letters" (in the Unicode sense); deprecating CHAR-ALPHABETIC? (which is esoteric at best, nonsense at worst); and defining the class of CHAR-ALPHABETIC? characters to be the largest subset of CHAR-LETTER? which is consistent with the R5RS definition. Now, having said all of that, the definition of CHAR-ALPHABETIC? could be improved: The possibilitiy of non-alphabetic letters with both upper and lowercase forms seems plausble to me (are there any in Unicode already?) So, instead of that definition of CHAR-ALPHABETIC? I would agree to: CHAR-ALPHABETIC? must be defined in such a way that this is true of all characters: (or (not (char-alphabetic? c)) (and (char-letter? c) (char-upper-case? (char-upcase c)) (char-lower-case? (char-downcase c)))) => #t Note: this requirement is necessary for a combination of upward compatability with earlier versions of the Revised Report and consistency with the new CHAR-LETTER?, yet it is also linguistically undesirable. This is the reason that CHAR-ALPHABETIC? is described as "deprecated" -- new programs should avoid using this procedure and should, in most cases, use CHAR-LETTER? instead. Programmers should be aware that the class CHAR-LETTER? may include letters such as syllables and ideographs which are not, in any sense, "alphabetic". It can also include alphabetic characters which are neither upper or lowercase, lowercase letters with no uppercase form, uppercase letters with no lowercase form, lowercase characters which are not returned by CHAR-DOWNCASE of their CHAR-UPCASE mapping, and uppercase charactes which are not returned by CHAR-UPCASE of their CHAR-DOWNCASE mapping. Programmers should also be aware that in some situations, a string may contain a letter followed by non-letters -- the sequence being "what a user would think of as a single letter" -- a fact which limits the utility of even CHAR-LETTER? unless additional facilities for text processing are provided by an implementation. Yet at the same time, for the portable character set and for many extended characters, none of these peculiar circumstances apply -- programmers not trying to write "fully general" text processing algorithms can often ignore these complexities. Programmers wanting to write "fully general" text algorithms, on the other hand, can define additional procedures which complement the standard character classes. > char-alphabetic? > char-numeric? > char-whitespace? > char-upper-case? > char-lower-case? > These procedures return #t if their arguments are alphabetic, > numeric, whitespace, uppercase, or lowercase characters, respectively. > Otherwise they return #f. The characters a..z and A..Z are required to > be alphabetic. The digits 0..9 must be numeric. The space, newline, and > tab characters must be whitespace. The characters a..z are required to > be lowercase. The characters A..Z are required to be uppercase. No > character may be both uppercase and lowercase. That's consistent with my proposed revisions. I think CHAR-LETTER? ought to be added and CHAR-ALPHABETIC? either dropped entirely or mentioned as deprecated. If it is mentioned as deprecated, the invariant shown above should be stated here. The corresponding sentence in the definition of CHAR-UPCASE and CHAR-DOWNCASE should be dropped. > char-cased? > char-uncased? > Char-cased? returns #t if its argument is a character which conforms to > "normal" case expectations, (see below) and #f otherwise. [....] > Rationale: This allows char-lower-case?, char-upper-case?, and > char-alphabetic? to go on meaning the same thing with respect to the > 96-character portable character set and meaning the same thing > linguists mean when they use these terms. This will reduce confusion > in the long run. This particular notion of cased and uncased > characters is also useful in other parts of the standard for saying > exactly which characters case requirements should apply to. It leaves > implementors free to not sweat about what to do with identifiers > containing eszett, regardless of what they do with calls to > (char-upcase #\eszett). Among the rationales: I think this one is false (see above): > This particular notion of cased and uncased characters is also > useful in other parts of the standard for saying exactly which > characters case requirements should apply to. The other rationales are are good reasons to say _something_ but I don't think two new procedures are needed. Instead, the possibilitiy of oddly-cased characters can be explicitly mentioned in the definitions of CHAR-LOWER-CASE?, CHAR-UPPER-CASE?, and CHAR-LETTER?. (Additionally, CASED and UNCASED seems like poor names for the classes of characters they describe.) >> With regard to [...] char-upcase and char-upcase >> It should say >> [....] char-upcase must map a..z to A..Z and >> char-downcase must map A..Z to a..z. > I would propose instead: > [...] if char is alphabetic and cased, then the result of > char-upcase is upper case and the result of char-downcase is > lower case. I'm not sure I see any value to the stronger requirement, especially since CHAR-ALPHABETIC? should be deprecated and there is otherwise no need to introduce the concept of a "cased" character. Your alternative is implied by the definition of CHAR-ALPHABETIC? I gave in the draft -- but you've earlier convinced me to weaken that definition. >> The introduction to strings [....] should say: >> Some of the procedures that operate on strings ignore the difference >> between strings in which upper and lower case variants of the same >> character occur in corresponding positions. The versions that ignore >> case have ``-ci'' (for ``case insensitive'') embedded in their >> names. > I would propose instead: > Some of the procedures that operate on strings ignore the difference > between upper and lower case cased characters. The versions that > ignore case in cased characters have ``-ci'' (for ``case > insensitive'') embedded in their names. I believe that this should be true: (char=? #\dotless-i #\U+0131) => #t (char-ci=? #\I #\dotless-i) => #t and that STRING-CI=? is just the string equivalence induced by CHAR-CI=?. However, #\dotless-i is not "cased" as you have defined it. Are you saying that #\dotless-i and #\I are not CHAR-CI=? or that STRING-CI=? is not the equivalence induced by CHAR-CI=?? Either way: why in the world do that? -t ---- Like my work on GNU arch, Pika Scheme, and other technical contributions to the public sphere? Show your support! https://www.paypal.com/xclick/business=lord%40emf.net&item_name=support+for+arch+and+other+free+software+efforts+by+tom+lord&no_note=1&tax=0¤cy_code=USD and lord@xxxxxxx for www.moneybookers.com payments.