On Fri, Apr 15, 2011 at 6:25 AM, Artyom <artyomtnk@yahoo.com> wrote:
In binary level, this is same.
What should we do? Using string literal with no prefix is practically dangerous.
You should do following.
1. Do not include direct localized text to sources, that is what localization system for. How can we force programmer for not using any character other than basic source character set? It's impossible. They write whatever they can. You need at least warn Not to use character other than basic source character set in the document. But still, people write it. Since this is a localization library. I bet all Japanese expect it can handle Japanese input and give us the mapped translation as the output.
2. If you have to then use UTF-8 in the sources that it would remain UTF-8
How many times do I have to explain? MSVC automatically convert encoding for the content of string literal with no encoding prefix. There is no way to prevent it unless you use proper encoding prefix u8 which most compiler doesn't support yet.
for char * strings as well and would be handled correctly for L"..." prefixes. But translate expect char.
3. When you use MSVC add UTF-8 BOM (even thou it is useless but this is the way MSVC distinguishes between local encodings and UTF-8)
Bad but this is reality.
In fact almost all Unit tests in Boost.Locale use inline UTF-8 and believe me, it made my life significantly simpler.
Your life may be simple. You don't need to worry about complicated Japanese environment. But I am not.
Artyom _______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
-- Ryou Ezoe