This page is part of the web mail archives of SRFI 75 from before July 7th, 2015. The new archives for SRFI 75 contain all messages, not just those from before July 7th, 2015.
Alan Watson writes: > Using UTF-8 internally for a Scheme on a Plan 9 system is not obviously > a bad idea. Sure, you don't have direct indexing, but you avoid > conversion when you talk to the C library and OS. True enough. > Using UTF-16 internally doesn't give you direct indexing because of > characters outside the BMP, but it might make sense on Windows boxes for > precisely the same reason. This is a valid point. Python took the view that by default UTF-16 is used internally then direct indexing into a string could yield part of a surrogate pair. The feeling (as I remember, I may be wrong) was that astral plane characters are rare-enough that the common-case (i.e., BMP) should not be penalized. > Using UCS-32 internally in these cases would involve translation to talk > to the library and OS and would further make my emacs use about four > times as much memory as it does now (which brings us back the the > representation for infinity). Yes, though the glibc folks decided that the wchar_t type be a 4-byte Unicode value. Python gives you the option of building with a 4-byte or 2-byte "Unicode" character. (In Python Unicode and "narrow" strings are separate types.) > In general, any single representation is a bad idea in some circumstances. Absolutely. -- Tom Emerson Basis Technology Corp. Software Architect http://www.basistech.com "Beware the lollipop of mediocrity: lick it once and you suck forever"