On 01/29/2011 05:30 AM, Corinna Vinschen wrote: >> But when characters outside the basic plane, such as >> U+12345 (CUNEIFORM SIGN URU TIMES KI), are encoded by 2 consecutive wchar_t >> values, values of type wchar_t don't correspond to ISO/IEC 10646 characters. >> (Or maybe I'm underestimating what "coded representations" means...?) > > I don't read that from your above quote. The core is that the *type* > wchar_t is a *coded* *representation* of the characters defined in > 10646. At no point it says that a single wchar_t value must represent a > single character from 10646. So I take it that UTF-16 is a valid, coded > representation of the characters from 10646.
POSIX is clear that wchar_t must be wide enough so that 1 wchar_t is one character. Which limits a 2-byte wchar_t to just the Unicode basic plane. There's nothing cygwin can do about this other than break LOTS of ABI to support a 4-byte wchar_t to supply all of Unicode. http://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap06.html#tag_06_03 "All wide-character codes in a given process consist of an equal number of bits. This is in contrast to characters, which can consist of a variable number of bytes. The byte or byte sequence that represents a character can also be represented as a wide-character code. Wide-character codes thus provide a uniform size for manipulating text data." So, using UTF-16 surrogate encodings for characters outside the basic plane violates POSIX, but it's the best we can do for those characters. > I've put a lot of effort in 2009 and early 2010 to make the wchar_t > representation in Cygwin and newlib as much Unicode 5.2 compatible as > possible. Even the wcrtomb and mbrtowc functions in newlib are capable > of dealing with UTF-16 surrogates. And I appreciate that effort - even though it means wchar_t is just as painful as multi-byte char characters in that an array of wchar_t is not necessarily that many characters long, but only when surrogates are involved. > > However, given that Windows XP basically only supports the charset from > Unicode 4.0, and given that Cygwin's support for east-asian double and > triple byte codesets (Big5, GBK, eucKR, eucJP, and a SJIS/CP932 bastard) > still requires the underlying Windows conversion functions, I've set > __STDC_ISO_10646__ to a value which reflects Unicode 4.0 (200305L) for > Cygwin 1.7.8. Someday when gcc has better support for C+1x 16- and 32-bit characters (regardless of the sizing of wchar_t), then we can add all the new 32-bit character APIs that use Unicode unimpeded, without breaking existing ones that use wchar_t. -- Eric Blake ebl...@redhat.com +1-801-349-2682 Libvirt virtualization library http://libvirt.org
signature.asc
Description: OpenPGP digital signature
