On Fri, Aug 12, 2011 at 15:29, Daniel James <dnljms@gmail.com> wrote:
[...] A part of the popularity of boost is because it works well with existing code. So we need to work with the strings we get at the command line, from streams etc.
It will work well with existing code. The user will just need to convert her strings to UTF-8 if they aren't UTF-8 already.
Unless a different solution can be found.
Exactly.
I meant a different solution to assuming that std::string is always UTF-8. It appears unlikely that your proposal will be accepted by boost, so what other possibilities are there? Perhaps a distinct string type, maybe some mechanism to specify what encoding strings are using, or something else entirely?
I assume that we all agree that we want to encourage UTF-8, as it's the only way to handle Unicode on all systems except windows. Assuming we *do* want to move to UTF-8, we have just one problem, compatibility. We cannot do both, use UTF-8 and don't break or change any existing code. It's just impossible. See (3) in my previous mail. (4) I've already proposed another solution on another thread. We can add a compile-time flag that, if set, makes all the narrow char and std::string interfaces assume UTF-8 encoding. This flag can be off by default as long as we feel comfortable. When more people find this useful we can make it the default and deprecate the non-UTF-8 configuration. This is *the only way* to kill two birds with one stone. Any other solution either silently breaks existing code, or requires boilerplate code to be added even in code that uses utf-8 std::strings. I think this is the simplest, least painful way to switch to UTF-8. In fact, if the experiment fails, we can just deprecate the feature, remove it later and pretend it'd never happened. Unfortunately, the community mostly ignored this proposal. The only reply from a library author showed complete unwillingness to soil his holy code with such unimportant things like portable Unicode support on windows. He sent me to request UTF-8 support from microsoft itself. -- Yakov