Daryle Walker <darylew@hotmail.com> writes:
I think that there should be at most two _external_ Unicode string types:
1. vector of Unicode code-points externally each element is like a int32_t 2. vector of abstract Unicode characters externally each element is a group of Unicode code-points (a primary or starting code point, followed by combiner codes)
Some users do care about the outside appearance. (I think some guy here wanted UTF-8 XML.) In those cases we have a specific input or output routine that uses an appropriate encoding object, to hide whether or not the Unicode string internally uses the same encoding as the final source/sink.
Iterators probably should be made for code points and/or abstract characters. Bidirectional travel would be easiest. Such iterators should be configured (at compile- and/or run-time) for various normalization schemes.
Input needs special handling, since we shouldn't allow ultimately invalid byte/code-point combinations into Unicode strings. We need something that can enumerate over a byte stream for a particular encoding and spit out whole code-points (or queue the code-points and spit out abstract characters).
The XML parser I have under development on Sourceforge (http://www.sf.net/projects/axemill) includes string handling facilities that support the above. I haven't yet found a need for dealing with "abstract characters", since the closest thing in XML (name matching) requires that the names use the same sequence of code points (including combining characters) in all places. The "vector of Unicode code-points" I use is a std::basic_string<UnicodeCharacter>, where UnicodeCharacter is a POD struct with a 32-bit int member to represent the unicode code point. I have to do it that way to allow customization of std::char_traits, since you cannot specialize std::char_traits for built-in types. Anthony -- Anthony Williams Software Developer