- What is your evaluation of the design? Functions. Seems straight forward to use. - What is your evaluation of the implementation? What I used worked. I did not look over the source. - What is your evaluation of the documentation? Good except that there are at least some errors/inconsistencies (ngettext in messages_formatting.html for example is missing an argument). Also need to be explicit about which version of 3rd party programs are required (see below). - What is your evaluation of the potential usefulness of the library? I'll probably be using it for our product (assuming it plays fine with Qt, which I don't see any reason why not) so I'd say very. - Did you try to use the library? With what compiler? Did you have any problems? Yes. MSVC2010. No problems specific to the library but did have a pretty hard time figuring out which version of gettext was necessary and finding it. Simple search for gettext win32 or windows does not turn up a version compatible with this library (--keyword=translate:1,1t syntax invalid). Had to get the recent MSYS developer toolkit (mingw + msys wasn't even enough). After a couple of hours of searching it seemed to be the only option. - How much effort did you put into your evaluation? A glance? A quick reading? In-depth study? I wrote a simple program and played with a few of the functions. I downloaded and installed various tools that are supposed to help with translating. Basically did a short, 4 hour evaluation of whether I could propose it for use. - Are you knowledgeable about the problem domain? Not as much as I certainly should be. - Should it be in boost? I have some concerns about the fact that it relies so much on 3rd party stuff. The gettext tools are obviously moving targets and not having an easily accessible version of xgettext for Visual Studio users is not an optimal situation. Having to install MinGW and the works shouldn't be required. Various supposed binary downloads of gettext for win32 either are not or are too old. I also found that of the tools listed for working on po files, only one both worked with plural forms and worked on win32. In fact possibly only the one worked with plural forms at all. The fact that this library implements a gettext like setup is nice, but the dependency then upon gettext to provide the source extraction tools and such is unfortunate and could certainly lead to problems where there are version conflicts. Based on my experience I'd say that clearly there are some availability issues in this area for those using Visual Studio. Since gettext is someone else's problem, I don't see how boost can safely make sure the necessary programs are available to anyone using boost. Certainly some better documentation about which version of gettext is being emulated should be included (I looked through the docs and never saw anything specifying version). Once I did find the correct version of the tools (or at least a version that responded as the documentation explained) the system seemed to work out just fine. I would like to see an alternative to a complete msys install (though I personally like to have it around, others on my team find it useless). I would guess that there should be one but I seriously couldn't find it. I would also like to see boost::locale::format work like boost::format. The documentation also calls this thing a printf like utility class but it doesn't, as opposed to boost::format, accept printf format strings. On the other hand, boost::format(translate("%.4g")) % 3.14 worked just fine so I don't know what this locale specific version is providing. All that said, I think it could be a good addition. We certainly need something like it and it at least does what I think I need from such a library.
On 4/15/2011 3:53 PM, Noah Roberts wrote:
- What is your evaluation of the design?
Functions. Seems straight forward to use.
- What is your evaluation of the implementation?
What I used worked. I did not look over the source.
Actually, I ran into what I would call a bug. #include <boost/locale.hpp> #include <boost/lexical_cast.hpp> #include <iostream> using namespace boost::locale; int main() { generator gen; std::locale::global(gen("fr_CA.UTF-8")); //std::locale::global(std::locale("")); std::string dbl = boost::lexical_cast<std::string>(3.14); std::cout << dbl << std::endl; std::cin.get(); } My language settings currently use ',' as decimal separator as do the settings for the french (Canada) language settings. The commented out version outputs "3,14000000001". This is what I would expect to happen. The boost::locale version outputs "3.14000000001". It looks to me like the punctuation stuff isn't being set up correctly. The as::number thing doesn't do anything either. Only locale::format catches on and does it right. I managed to get it going by hacking around a bit. Not sure if what I'm doing is actually close to anything that might be what it should be... std::locale loc0(gen("")); std::locale loc1(""); std::locale::global(std::locale(loc0, loc1, std::locale::all)); I'd expect that the delimiters and such would be set up the same in both cases and this merge attempt of mine shouldn't be necessary.
----- Original Message ----
From: Noah Roberts <roberts.noah@gmail.com>
Actually, I ran into what I would call a bug.
#include <boost/locale.hpp> #include <boost/lexical_cast.hpp>
#include <iostream>
using namespace boost::locale;
int main() { generator gen;
std::locale::global(gen("fr_CA.UTF-8")); //std::locale::global(std::locale(""));
std::string dbl = boost::lexical_cast<std::string>(3.14); std::cout << dbl << std::endl;
std::cin.get(); }
My language settings currently use ',' as decimal separator as do the settings for the french (Canada) language settings.
The commented out version outputs "3,14000000001". This is what I would expect to happen. The boost::locale version outputs "3.14000000001".
It looks to me like the punctuation stuff isn't being set up correctly. The as::number thing doesn't do anything either. Only locale::format catches on and does it right.
I'll explain, it is not a bug it is a feature. 1st of all read this: http://cppcms.sourceforge.net/boost_locale/html/std_locales.html http://cppcms.sourceforge.net/boost_locale/html/std_locales.html#std_locales... Boost.Locale preserves default formatting as C/POSIX locale and it is very important. I've seen many libraries become broken because of setting gobal locale, consider cvs generation. So in order to generate localized number you should use as::number and in your case you also missed imbuing a locale object to cout. So by default numbers generated in "C" format so you sql library that binds sql << "SELECT values from FOO where bar > ?", 3.14; // SOCI example would not be broken, because the statement would be "SELECT values from FOO where bar > 3,14" in some locales. And I've seen lots of code that fails in such case just because 95% of programmers unaware of the fact the number may be localized. The problem with C even more severe, because unlike C++, C libraries have no option to imbue a locale and setting global C++ locale std::locale::globale(std::locale("")) Would call setlocale(LC_ALL,""); And it would totally mess C libraries that use sprintf to format numbers and get "," instead of "." So if you need localized numbers you should use as::number otherwise C/POSIX locale is used by default. It is by design. Artyom
On 16.04.2011 12:45, Artyom wrote:
[...] I've seen many libraries become broken because of setting global locale
We shouldn't adapt for broken 3d-party libraries. If you so like the current Boost.Locale's behavior, provide a special *care_with_broken_c_libs_generator*, that's not default.
So if you need localized numbers you should use as::number otherwise C/POSIX locale is used by default.
inconsistent requirement
It is by design.
it's wrong -- - Do you speak English? Мужик с глубоким вздохом: - Yes I do. А хули толку?
We shouldn't adapt for broken 3d-party libraries.
If you so like the current Boost.Locale's behavior, provide a special *care_with_broken_c_libs_generator*, that's not default.
So if you need localized numbers you should use as::number otherwise C/POSIX locale is used by default.
inconsistent requirement
It is by design.
it's wrong
If so please implement following in C correctly: void delete_values_above_x(double x) { char statement[256]; snprintf(statement,sizeof(statement),"DELETE FROM sometable WHERE x > %f",x); mysql_query(connection,statement) } When in current locale the decimal point is "," and x=3.14... You can't do it in reasonable simple way in C. It is design issue of standard C and C++ library. Artyom
Artyom wrote:
If so please implement following in C correctly:
void delete_values_above_x(double x) { char statement[256]; snprintf(statement,sizeof(statement),"DELETE FROM sometable WHERE x > %f",x); mysql_query(connection,statement) }
When in current locale the decimal point is "," and x=3.14...
You can't do it in reasonable simple way in C. It is design issue of standard C and C++ library.
Artyom
While I agree about the standard C library, what issue does C++ standard library have here? You can just imbue std::locale::classic() and you are good to go. Am I missing something? Gevorg
From: Gevorg Voskanyan <v_gevorg@yahoo.com> To: boost@lists.boost.org Sent: Mon, April 18, 2011 4:06:59 PM Subject: Re: [boost] [locale] Review
Artyom wrote:
If so please implement following in C correctly:
void delete_values_above_x(double x) { char statement[256]; snprintf(statement,sizeof(statement),"DELETE FROM sometable WHERE x > %f",x); mysql_query(connection,statement) }
When in current locale the decimal point is "," and x=3.14...
You can't do it in reasonable simple way in C. It is design issue of standard C and C++ library.
Artyom
While I agree about the standard C library, what issue does C++ standard library
have here? You can just imbue std::locale::classic() and you are good to go. Am I missing
something?
Calling std::locale::global(some_locale) calls C function setlocale(LC_ALL,some_locale.name()); So even if you can fix C++ code you can't do it for C libraries you may use. Also 99.9% of C++ developers unaware of this issue and almost nobody imbues std::locale::classic() because they just not aware of the problem. Artyom
Artyom wrote:
Gevorg Voskanyan wrote:
Artyom wrote:
It is design issue of standard C and C++ library.
While I agree about the standard C library, what issue does C++ standard library
have here? You can just imbue std::locale::classic() and you are good to go. Am I missing
something?
Calling std::locale::global(some_locale) calls C function
setlocale(LC_ALL,some_locale.name());
So even if you can fix C++ code you can't do it for C libraries you may use.
Also 99.9% of C++ developers unaware of this issue and almost nobody imbues std::locale::classic() because they just not aware of the problem.
Valid points, yet none of these is a design issue of the C++ standard library per se. You can write the example you mentioned earlier in this thread in a correct and reasonable way in C++, unlike in C. But developers need to be educated how, and I think this library can help with that by increasing the general awareness of localization-related issues in the C++ community. Gevorg
Gevorg wrote:
Calling std::locale::global(some_locale) calls C function
setlocale(LC_ALL,some_locale.name());
So even if you can fix C++ code you can't do it for C libraries you may use.
Also 99.9% of C++ developers unaware of this issue and almost nobody imbues std::locale::classic() because they just not aware of the problem.
Valid points, yet none of these is a design issue of the C++ standard library
per se. You can write the example you mentioned earlier in this thread in a correct and reasonable way in C++, unlike in C. But developers need to be educated how, and I think this library can help with that by increasing the general awareness of localization-related issues in the C++ community.
There are two problems: 1. Every medium size C++ project uses C libraries. And they will always be there. 2. When C++ locale's system is quite broken in its current state and useless for most of users then you can't expect them to be aware of such issues. The other problem is that I hear complains about it too frequently: - libdbi mailing list is full of it and there is open bug - SOCI has issues - The problem with lexical cast rise once in a while in Boost's list. No, if something causes bugs too frequently then there is something wrong with the concept. Artyom
Artyom wrote:
There are two problems:
1. Every medium size C++ project uses C libraries. And they will always be there. 2. When C++ locale's system is quite broken in its current state and useless for most of users then you can't expect them to be aware of such issues.
The other problem is that I hear complains about it too frequently:
- libdbi mailing list is full of it and there is open bug - SOCI has issues - The problem with lexical cast rise once in a while in Boost's list.
No, if something causes bugs too frequently then there is something wrong with the concept.
Artyom
So your design wrt. this is: nothing is localized unless explicitly requested to. That avoids having something accidentally localized when you don't want that, e.g. csv, serialization, some network protocol, whatever. But that is at the expense of making a way for the opposite kind of bugs: when something is not localized but should be. For example what does 02/03/2011 mean. Imagine a program where all dates in other places that are shown to the user are properly localized, but this one isn't due to a programmer omission. The result is, well, a very angry user with all its consequences. I'm not going to argue which one is more important. We lose something either way. And this concern is what the reviewers had in mind when they brought the issue up in this thread. In this respect yes, C++ standard library has design issue, but Boost.Locale has one too. It's a design choice, which happens to be different between C++ std lib and your lib. And I have no problem with that. But the documentation can discuss this issue in detail, describe common traps and the ways to avoid them. It can also have a general advice: if you want to localize your software, pay extra attention to what you want to be localized and what you don't. Gevorg
On 2011-04-16 01:45, Artyom wrote:
----- Original Message ----
From: Noah Roberts<roberts.noah@gmail.com>
Actually, I ran into what I would call a bug.
#include<boost/locale.hpp> #include<boost/lexical_cast.hpp>
#include<iostream>
using namespace boost::locale;
int main() { generator gen;
std::locale::global(gen("fr_CA.UTF-8")); //std::locale::global(std::locale(""));
std::string dbl = boost::lexical_cast<std::string>(3.14); std::cout<< dbl<< std::endl;
std::cin.get(); }
My language settings currently use ',' as decimal separator as do the settings for the french (Canada) language settings.
The commented out version outputs "3,14000000001". This is what I would expect to happen. The boost::locale version outputs "3.14000000001".
It looks to me like the punctuation stuff isn't being set up correctly. The as::number thing doesn't do anything either. Only locale::format catches on and does it right.
It is by design.
Then what is the correct work-around to get past this "design"? Some people, like me, make frequent use of boost::format (which your version cannot replace because it does not implement 'g' formatting), and other functions that output in characters specific to the locale and know nothing about as::anything. This is a problem. The library, perhaps rightly, doesn't follow standard conventions but it also doesn't replicate those functions which do. Can I use only your translation functionality without imbuing the streams with your locale? Something tells me that won't work out quite right. If this design is going to stick, then there certainly needs to be a documented workaround so that those of us who need the behavior can get it.
My language settings currently use ',' as decimal separator as do the
settings
for the french (Canada) language settings.
The commented out version outputs "3,14000000001". This is what I would expect to happen. The boost::locale version outputs "3.14000000001".
It looks to me like the punctuation stuff isn't being set up correctly. The as::number thing doesn't do anything either. Only locale::format catches on and does it right.
It is by design.
Then what is the correct work-around to get past this "design"? Some people, like me, make frequent use of boost::format (which your version cannot replace because it does not implement 'g' formatting), and other functions that output in characters specific to the locale and know nothing about as::anything.
I see much more complains about the other way around when people accidentially getting csv files like 3,435, 3,456, 3,345 And don't understand why.
This is a problem. The library, perhaps rightly, doesn't follow standard conventions but it also doesn't replicate those functions which do. Can I use only your translation functionality without imbuing the streams with your locale?
Yes you can use *gettext API and pass locale explicitly, you can also install message facets only and not formatting ones (see generator reference documentation).
Something tells me that won't work out quite right.
If this design is going to stick, then there certainly needs to be a documented workaround so that those of us who need the behavior can get it.
There are two workarounds: 1. Use boost::locale::as::number manipulator. 2. Use locale you want as base locale in generator: gen.get(std::locale("de_DE.UTF-8"),"de_DE.UTF-8"); // the base is C++ librarie's default one. But I really do not recommend you doing anything like that. Artyom
On 19.04.2011 1:25, Artyom wrote:
I see much more complains about the other way around when people accidentially getting csv files like
3,435, 3,456, 3,345
And don't understand why.
:) this is csv problem, not locale! Your reasoning [in http://cppcms.sourceforge.net/boost_locale/html/std_locales.html#std_locales...] is incorrect -- - Do you speak English? Мужик с глубоким вздохом: - Yes I do. А хули толку?
- What is your evaluation of the documentation?
Good except that there are at least some errors/inconsistencies (ngettext in messages_formatting.html for example is missing an argument).
Already noted, will be fixed.
- Did you try to use the library? With what compiler? Did you have any problems?
Yes. MSVC2010. No problems specific to the library but did have a pretty hard time figuring out which version of gettext was necessary and finding it. Simple search for gettext win32 or windows does not turn up a version compatible with this library (--keyword=translate:1,1t syntax invalid). Had to get the recent MSYS developer toolkit (mingw + msys wasn't even enough). After a couple of hours of searching it seemed to be the only option.
[snip]
I have some concerns about the fact that it relies so much on 3rd party
stuff.
The gettext tools are obviously moving targets and not having an easily accessible version of xgettext for Visual Studio users is not an optimal situation. Having to install MinGW and the works shouldn't be required. Various supposed binary downloads of gettext for win32 either are not or are too old.
I also found that of the tools listed for working on po files, only one both worked with plural forms and worked on win32 In fact possibly only the one worked with plural forms at all.
The fact that this library implements a gettext like setup is nice, but the dependency then upon gettext to provide the source extraction tools and such is unfortunate and could certainly lead to problems where there are version conflicts. Based on my experience I'd say that clearly there are some availability issues in this area for those using Visual Studio. Since gettext is someone else's problem, I don't see how boost can safely make sure the necessary programs are available to anyone using boost. Certainly some better documentation about which version of gettext is being emulated should be included (I looked through the docs and never saw anything specifying version).
I'll explain why I relate on 3rd part programs. It is not only xgettext it is much more, you will need many programs. - Gettext provides tools for extraction, merging catalogs and handling them. They are quite complex as they allow for example mark slightly changes source strings as Fuzzy and allow translater to fix them in much easier way. Reinventing this wheel would require lots of work and I'm not so sure it is justified. I admit that as Linux user I sometimes assume that getting latest 3rd part tools is a piece of cake and forget that for Windows much more explicit description required. I'll add to the tutorial pointer to the latest version of gettext tools for Windows. - You will also need tools like Lokalize, Poedit or other translation tools. They are very-very important. For example it took me half an hour to translate entire blog system with Lokalize to Hebrew. Why? Because it gives you embedded spell checker, allows to browse over untranslated strings, search specific strings easily (when I wanted to replace one term by other all over the strings) So translation tools will always be requirement. There are many of them and they are important and well designed. It was essential to replace the runtime part due to - License restrictions (LGPL) - Inability to handle multiple locales in same process. But other then that I don't think that it is essential to rewrite good tools unless you have something to gain. So I'll will add very explicit instructions on how to get latest GNU Gettext for Windows. And how to use older versions and what are their limitations.
Once I did find the correct version of the tools (or at least a version that responded as the documentation explained) the system seemed to work out just fine. I would like to see an alternative to a complete msys install (though I personally like to have it around, others on my team find it useless). I would guess that there should be one but I seriously couldn't find it.
I think it would be possible to provide gettext runtime installs outside boost project. And I'll also put an investigation about it.
I would also like to see boost::locale::format work like boost::format. The documentation also calls this thing a printf like utility class but it doesn't, as opposed to boost::format, accept printf format strings. On the other hand, boost::format(translate("%.4g")) % 3.14 worked just fine so I don't know what this locale specific version is providing.
Small differences between (A) out << boost::format(translate("%.4g")) % 3.14 And (B) out << boost::locale::format(translate("{1,num,p=4}")) % 3.14 A uses global locale while B uses out's locale for both transation and formatting. A would use display non-localized bumber (i.e. C locale) while B localized one like 3,14 instead of 3.14 There are huge differences between them and I'm not sure it is easy to close the gap without reimplementing boost::format.
All that said, I think it could be a good addition. We certainly need something like it and it at least does what I think I need from such a library.
Thanks Artyom
participants (4)
-
Artyom -
Gevorg Voskanyan -
Max Sobolev -
Noah Roberts