[iostreams][io_state_savers] Request for interest in some ios_base basic utilities
Hi, I'm a beginner of the standard IOStream library. While restructuring Boost.Chrono I/O I have find some classes that could be of general use (if the design is correct, of course). The first one is quite simple, is used to define manipulators with parameters. Instead of defining a class and the correspondent stream inserters and extractors the manip template class do all this at once. It can be used as follows class mendl: public manip<mendl> { public: explicit mendl(size_t how_many) : count(how_many) {} template <typename out_stream> void operator()(out_stream &out) const { for (size_t line = 0; line < count; ++line) { out.put(out.widen('\n')); } out.flush(); } private: size_t count; }; Now mendl can be use to insert n new lines at once as follows cout << mendl(3); The second one is an ios specific smart pointer ios_base_state_ptr (based on the use of ios_base::pword and ios_base::xalloc). The class ios_base_state is a ios_base_state_ptr that can not be null. It can be used like struct MyData { std::string time_fmt; std::string duration_fmt; }; std::string get_time_fmt(std::ios_base & ios) { return ios_base_state<MyData>(ios)->time_fmt; } ios_base_state stores a new MyData in the ios if not already stored and provides an smart pointer interface to it. The 3rd one (ios_base_flags) uses the ios_base::iword and xalloc to provide access to specific ios flags. It can be used as follows: class fmt_masks : ios_base_flags<fmt_masks> { typedef ios_base_flags<fmt_masks> base_type; public: fmt_masks(std::ios_base& ios): base_type(ios) {} enum type { uses_symbol = 1 << 0, uses_local = 1 << 1 }; }; Usage of this class can be duration_style::type get_duration_style(std::ios_base& ios) { return (fmt_masks(ios).flags() & fmt_masks::uses_symbol) ? duration_style::symbol : duration_style::prefix; } The interface and the implementation of these classes is attached. Do you see a design flag on these classes? If not, do you think these kind of utilities could be added to boost? If yes, could these be merged with Boost.io_state_savers or Boost.IOStreams to form a larger io utilities libraries? Best, Vicente
Hello! Actually such utilities are and would be very useful. The stateful manipulator is rather something simple however the second part the "state saver" is something very important. In fact today Boost.DateTime uses for example locale facet to provide such information which is wrong by design. If such state saver was around when it created it would be much better. So my +1 and big one. Few points. ------------ I had already implemented state saver for Boost.Locale but it is hidden in the sources, it has a little bit different approach but in general it is very similar. https://svn.boost.org/svn/boost/trunk/libs/locale/src/shared/ios_prop.hpp I'd also suggest to add std::ios_base::imbue_event handing because it may be actually important. (Consider some caching withing object) Now some important (extremely important) notes about this code: --------------------------------------------------------------- static inline int index() { static const int v_ = std::ios_base::xalloc(); return v_; } 1. This code may be not thread safe and may create two different values for two different threads. So I'd suggest to provide some global initialization option to at least illeminate somehow this problem or to use Boost.Thread's ones initialization. 2. This code can't be used on cross dll boundaries. Consider you have function (non-linline) boost::foo::bar in libboost_foo.dll that updates manipulator in the exe other inline function boost::foo::baz uses it. You'll be surprised what will happen: the "static const int v_" in the index() function would have two instances and would allocate two distinct indexes. 3. You should never use it with static linked runtime and dlls (of course) because there will be two instances of xalloc()'s counter. But this is related to gcc 4.4 and below on Windows only. So library should be very careful and not use such smart pointer in inline instances and this should be written with big red letters in the top of the library docs. That is why Boost.Locale's ios_prop.hpp is hidden in sources. (Yep... dll platforms sucks... :-) or make our work a way much harder) Artyom Beilis -------------- CppCMS - C++ Web Framework: http://cppcms.sf.net/ CppDB - C++ SQL Connectivity: http://cppcms.sf.net/sql/cppdb/
________________________________ From: Vicente J. Botet Escriba <vicente.botet@wanadoo.fr>
Hi,
I'm a beginner of the standard IOStream library. While restructuring Boost.Chrono I/O I have find some classes that could be of general use (if the design is correct, of course).
The first one is quite simple, is used to define manipulators with parameters. Instead of defining a class and the correspondent stream inserters and extractors the manip template class do all this at once.
[snip]
Do you see a design flag on these classes? If not, do you think these kind of utilities could be added to boost? If yes, could these be merged with Boost.io_state_savers or Boost.IOStreams to form a larger io utilities libraries?
Best, Vicente
_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Artyom Beilis wrote:
Hello!
Actually such utilities are and would be very useful.
The stateful manipulator is rather something simple
I agree, very simple but so easy to use :)
however the second part the "state saver" is something very important.
In fact today Boost.DateTime uses for example locale facet to provide such information which is wrong by design. If such state saver was around when it created it would be much better.
Yes, Boost.Chrono I/O V1 is using this design also. I'm creating a Boost.Chrono v2 design that stores information like duration style, timezone, time format in the ios state. The inconvenient is the thread safety and inter-module interferences. I think that the locale/facet approach doesn't suffer from these thread and inter-module issues. What do you think? Anyway the IOStream library is not thread safe, so the user can not use them without falling in undefined behavior. The inter-module issue is another history.
So my +1 and big one.
Few points. ------------
I had already implemented state saver for Boost.Locale but it is hidden in the sources, it has a little bit different approach but in general it is very similar.
https://svn.boost.org/svn/boost/trunk/libs/locale/src/shared/ios_prop.hpp
Glad to see others have taken the same approach. I will not call that a state saver, just a state.
I'd also suggest to add std::ios_base::imbue_event handing because it may be actually important. (Consider some caching withing object)
Yes, all the callback events shall be taken in account.
Now some important (extremely important) notes about this code: ---------------------------------------------------------------
static inline int index() { static const int v_ = std::ios_base::xalloc(); return v_; }
1. This code may be not thread safe and may create two different values for two different threads.
So I'd suggest to provide some global initialization option to at least illeminate somehow this problem or to use Boost.Thread's ones initialization.
I agree, the code is not thread safe.
2. This code can't be used on cross dll boundaries.
Consider you have function (non-linline) boost::foo::bar in libboost_foo.dll that updates manipulator in the exe other inline function boost::foo::baz uses it.
You'll be surprised what will happen: the "static const int v_" in the index() function would have two instances and would allocate two distinct indexes.
I know these issues. The final implementation would need a kind of thread safe and multi DLL singleton, maybe based on the holder included in Boost.Flyweight.
3. You should never use it with static linked runtime and dlls (of course) because there will be two instances of xalloc()'s counter.
But this is related to gcc 4.4 and below on Windows only.
So library should be very careful and not use such smart pointer in inline instances and this should be written with big red letters in the top of the library docs.
Well I see two alternatives here: either the library state that this can be used in a single threaded program and in a single module, or it is thread and intermodule safe.
That is why Boost.Locale's ios_prop.hpp is hidden in sources.
How the fact to hide it makes it more thread or module safe. Could you clarify this?
(Yep... dll platforms sucks... :-) or make our work a way much harder)
________________________________ From: Vicente J. Botet Escriba <vicente.botet@>
Hi,
I'm a beginner of the standard IOStream library. While restructuring Boost.Chrono I/O I have find some classes that could be of general use (if
Yes, and unfortunately this is something we need to live with :( Best, Vicente the design is correct, of course).
The first one is quite simple, is used to define manipulators with
parameters. Instead of defining a class and the correspondent stream inserters and extractors the manip template class do all this at once.
[snip]
Do you see a design flag on these classes? If not, do you think these kind of utilities could be added to boost? If yes, could these be merged with Boost.io_state_savers or Boost.IOStreams
to form a larger io utilities libraries? -- View this message in context: http://boost.2283326.n4.nabble.com/iostreams-io-state-savers-Request-for-int... Sent from the Boost - Dev mailing list archive at Nabble.com.
----- Original Message -----
From: Vicente Botet <vicente.botet@wanadoo.fr>
however the second part the "state saver" is something very important.
In fact today Boost.DateTime uses for example locale facet to provide such information which is wrong by design. If such state saver was around when it created it would be much better.
Yes, Boost.Chrono I/O V1 is using this design also. I'm creating a Boost.Chrono v2 design that stores information like duration style, timezone, time format in the ios state.
On thing - boost.locale already does all this. I mean there is manipulators to handle time zone, time formats and other things. I'd suggest to take a look on them. Because this may be all you need. Also it can be extended with some new styles easily if needed like for example "duration" style. So I'd suggest to use them.
The inconvenient is the thread safety and inter-module interferences. I think that the locale/facet approach doesn't suffer from these thread and inter-module issues. What do you think?
No, it has exactly the same problem... The locale facet approach is not correct for other reason. Locale/facet object should hold general rules and methods of handling data and its facets are shared among multiple instances of std::(i|o)stream objects. For that reason std::use_facet<Foo> returns Foo const & and not mutable reference. Now iostream flags should hold (and hold) user specific data like how to format date/time, number according to rules and they are stream specific not locale specific.
Anyway the IOStream library is not thread safe, so the user can not use them without falling in undefined behavior.
Actually most iostream implementations are thread safe - not in terms that same object can be accessed from two threads but rather things that are expected to be accessed from different threads like xalloc. For example xalloc is thread safe in both MSVC and GCC libraries.
So my +1 and big one.
Now some important (extremely important) notes about this code: ---------------------------------------------------------------
static inline int index() { static const int v_ = std::ios_base::xalloc(); return v_; }
1. This code may be not thread safe and may create two different values for two different threads.
So I'd suggest to provide some global initialization option to at least illeminate somehow this problem or to use Boost.Thread's ones initialization.
I agree, the code is not thread safe.
2. This code can't be used on cross dll boundaries.
Consider you have function (non-linline) boost::foo::bar in libboost_foo.dll that updates manipulator in the exe other inline function boost::foo::baz uses it.
You'll be surprised what will happen: the "static const int v_" in the index() function would have two instances and would allocate two distinct indexes.
I know these issues. The final implementation would need a kind of thread safe and multi DLL singleton, maybe based on the holder included in Boost.Flyweight.
I don't know if chrono is header only library or not. Just you need to make sure that such library can't be header only. As long as it is not header only there will be no such problem.
3. You should never use it with static linked runtime and dlls (of course) because there will be two instances of xalloc()'s counter.
But this is related to gcc 4.4 and below on Windows only.
So library should be very careful and not use such smart pointer in inline instances and this should be written with big red letters in the top of the library docs.
Well I see two alternatives here: either the library state that this can be used in a single threaded program and in a single module, or it is thread and intermodule safe.
Yes, unfortunately it is the limitation.
That is why Boost.Locale's ios_prop.hpp is hidden in sources.
How the fact to hide it makes it more thread or module safe. Could you clarify this?
Because, I can initialize all at once and don't live to user an option to choose when to do it. However once all "static" variables are ready there no more thread issues. For example all locale ids and all xalloc ids are initialized once globally.
(Yep... dll platforms sucks... :-) or make our work a way much harder)
Yes, and unfortunately this is something we need to live with :(
Best, Vicente
Regards, Artyom Beilis -------------- CppCMS - C++ Web Framework: http://cppcms.sf.net/ CppDB - C++ SQL Connectivity: http://cppcms.sf.net/sql/cppdb/
Artyom Beilis wrote:
----- Original Message -----
From: Vicente Botet <vicente.botet@>
however the second part the "state saver" is something very important.
In fact today Boost.DateTime uses for example locale facet to provide such information which is wrong by design. If such state saver was around when it created it would be much better.
Yes, Boost.Chrono I/O V1 is using this design also. I'm creating a Boost.Chrono v2 design that stores information like duration style, timezone, time format in the ios state.
On thing - boost.locale already does all this. I mean there is manipulators to handle time zone, time formats and other things. I'd suggest to take a look on them. Because this may be all you need.
Also it can be extended with some new styles easily if needed like for example "duration" style.
So I'd suggest to use them.
I'll take a look, but I don't think that I will be able to use the Boost.Locale manipulators, as the intents is to make a proposal for the standard, so dependence on other libraries should be minimized at least at the interface level.
The inconvenient is the thread safety and inter-module interferences. I think that the locale/facet approach doesn't suffer from these thread and inter-module issues. What do you think?
No, it has exactly the same problem...
OK.
The locale facet approach is not correct for other reason.
Locale/facet object should hold general rules and methods of handling data and its facets are shared among multiple instances of std::(i|o)stream objects.
For that reason std::use_facet<Foo> returns Foo const & and not mutable reference.
Now iostream flags should hold (and hold) user specific data like how to format date/time, number according to rules and they are stream specific not locale specific.
I agree. This is the reason I'm preparing a chrono I/O v2.
Anyway the IOStream library is not thread safe, so the user can not use them without falling in undefined behavior.
Actually most iostream implementations are thread safe - not in terms that same object can be accessed from two threads but rather things that are expected to be accessed from different threads like xalloc.
For example xalloc is thread safe in both MSVC and GCC libraries.
OK, I see. What I don't see is how a library such as Boost.Chrono using standard libraries that are not ensured to be thread safe, can be thread safe other than, as Boost.Locale does, force the initialization of all these data subject to race condition to be initialized before any thread is launched. I don't like this kind of design, which forces the user to initialize the library. I will expect we can do better.
So my +1 and big one.
Now some important (extremely important) notes about this code: ---------------------------------------------------------------
static inline int index() { static const int v_ = std::ios_base::xalloc(); return v_; }
1. This code may be not thread safe and may create two different values for two different threads.
So I'd suggest to provide some global initialization option to at least illeminate somehow this problem or to use Boost.Thread's ones initialization.
I agree, the code is not thread safe.
2. This code can't be used on cross dll boundaries.
Consider you have function (non-linline) boost::foo::bar in libboost_foo.dll that updates manipulator in the exe other inline function boost::foo::baz uses it.
You'll be surprised what will happen: the "static const int v_" in the index() function would have two instances and would allocate two distinct indexes.
I know these issues. The final implementation would need a kind of thread safe and multi DLL singleton, maybe based on the holder included in Boost.Flyweight.
I don't know if chrono is header only library or not.
Just you need to make sure that such library can't be header only. As long as it is not header only there will be no such problem.
Well I spent some time to make Boost.Chrono header only. I will be a little bit disappointed if I can not make a library that uses standard I/O thread safe other than using some tricks that can not be used in a header only library. ...snip...
That is why Boost.Locale's ios_prop.hpp is hidden in sources.
How the fact to hide it makes it more thread or module safe. Could you clarify this?
Because, I can initialize all at once and don't live to user an option to choose when to do it.
However once all "static" variables are ready there no more thread issues.
For example all locale ids and all xalloc ids are initialized once globally.
Beside the static data initialization, how do you ensure the access (R or W) to the ios state flags and locale facets are thread safe? Are you using a mutex? Best, Vicente -- View this message in context: http://boost.2283326.n4.nabble.com/iostreams-io-state-savers-Request-for-int... Sent from the Boost - Dev mailing list archive at Nabble.com.
From: Vicente Botet <vicente.botet@wanadoo.fr>
Artyom Beilis wrote:
----- Original Message ----- Also it can be extended with some new styles easily if needed like for example "duration" style.
So I'd suggest to use them.
I'll take a look, but I don't think that I will be able to use the Boost.Locale manipulators, as the intents is to make a proposal for the standard, so dependence on other libraries should be minimized at least at the interface level.
Actually I'm planning to do some proposals with Boost.Locale but I see your point.
For example xalloc is thread safe in both MSVC and GCC libraries.
I don't like this kind of design, which forces the user to initialize the library. I will expect we can do better.
Global constructor?
I don't know if chrono is header only library or not.
Just you need to make sure that such library can't be header only. As long as it is not header only there will be no such problem.
Well I spent some time to make Boost.Chrono header only. I will be a little bit disappointed if I can not make a library that uses standard I/O thread safe other than using some tricks that can not be used in a header only library.
The problem is rather inter module access rather then thread safety issue. And inter module access (cross dll) you can't do without library part under DLL platform... And singleton can't be done in header only library under dll platform as well. So... It is problem. However it is not problem for standard submission as you always have library - standard c++ library wich by no means can be header only.
For example all locale ids and all xalloc ids are initialized once globally.
Beside the static data initialization, how do you ensure the access (R or W) to the ios state flags and locale facets are thread safe? Are you using a mutex?
Locale facets, once installed are immutable objects. So they become thread safe by immutability(*) as they either operate on constant data. Now ios state flags should not be thread safe as they stream specific and I don't expect from user to alter iostream flags from different threads. (*) With ICU not all facets are immutable so in some places I use TLS to improve performance as for example with collator.
Best, Vicente
Best, Artyom Beilis -------------- CppCMS - C++ Web Framework: http://cppcms.sf.net/ CppDB - C++ SQL Connectivity: http://cppcms.sf.net/sql/cppdb/
Artyom Beilis wrote:
From: Vicente Botet <vicente.botet@>
Artyom Beilis wrote:
----- Original Message ----- Also it can be extended with some new styles easily if needed like for example "duration" style.
So I'd suggest to use them.
I'll take a look, but I don't think that I will be able to use the Boost.Locale manipulators, as the intents is to make a proposal for the standard, so dependence on other libraries should be minimized at least at the interface level.
Actually I'm planning to do some proposals with Boost.Locale but I see your point.
For example xalloc is thread safe in both MSVC and GCC libraries.
I don't like this kind of design, which forces the user to initialize the library. I will expect we can do better.
Global constructor?
I don't see what you mean.
I don't know if chrono is header only library or not.
Just you need to make sure that such library can't be header only. As long as it is not header only there will be no such problem.
Well I spent some time to make Boost.Chrono header only. I will be a little bit disappointed if I can not make a library that uses standard I/O thread safe other than using some tricks that can not be used in a header only library.
The problem is rather inter module access rather then thread safety issue. And inter module access (cross dll) you can't do without library part under DLL platform...
And singleton can't be done in header only library under dll platform as well.
Isn't Boost.Flyweight a header only library?
So... It is problem. However it is not problem for standard submission as you always have library - standard c++ library which by no means can be header only.
Well for the standard submission there is thread or module issue if this state is included directly on the ios state and the default facet is already imbue with the associated facets ;-). Of course this is something I cannot do with Boost.Chorno, or I don't know how to do it :'(.
For example all locale ids and all xalloc ids are initialized once globally.
Beside the static data initialization, how do you ensure the access (R or W) to the ios state flags and locale facets are thread safe? Are you using a mutex?
Locale facets, once installed are immutable objects. So they become thread safe by immutability(*) as they either operate on constant data.
Yes I see. Only the facet id is concerned by multi-threading.
Now ios state flags should not be thread safe as they stream specific and I don't expect from user to alter iostream flags from different threads.
(*) With ICU not all facets are immutable so in some places I use TLS to improve performance as for example with collator.
Why do you need mutable facets? Isn't this against the locale/facet design? Best, Vicente -- View this message in context: http://boost.2283326.n4.nabble.com/iostreams-io-state-savers-Request-for-int... Sent from the Boost - Dev mailing list archive at Nabble.com.
From: Vicente Botet <vicente.botet@wanadoo.fr>
For example xalloc is thread safe in both MSVC and GCC libraries.
I don't like this kind of design, which forces the user to initialize the library. I will expect we can do better.
Global constructor?
I don't see what you mean.
I mean that all static ids are installed in global constructor like: src/foo.cpp int get_id() { static int v = std::ios_base::xalloc(); return v; } namespace details { struct installer { installer() { get_id(); } } instance; } This makes sure that if somebody links with the library the static ids (facet's ids) and static variables allocated with xalloc are installed. Of course this would not solve problem when two threads try dynamically load the same library linked with this one. But this is quite extreme case.
And singleton can't be done in header only library under dll platform as well.
Isn't Boost.Flyweight a header only library?
I'm not familiar with it. So I don't know.
So... It is problem. However it is not problem for standard submission as you always have library - standard c++ library which by no means can be header only.
Well for the standard submission there is thread or module issue if this state is included directly on the ios state and the default facet is already imbue with the associated facets ;-). Of course this is something I cannot do with Boost.Chorno, or I don't know how to do it :'(.
But you can fallback to default if facet is not installed, i.e.: if(!have_facet<foo>(loc)) // Do something as std::locale::classic dooes else use_facet<foo>(loc).bar(...) Same for configuration. Ugly but useful for high level functions.
Now ios state flags should not be thread safe as they stream specific and I don't expect from user to alter iostream flags from different threads.
(*) With ICU not all facets are immutable so in some places I use TLS to improve performance as for example with collator.
Why do you need mutable facets? Isn't this against the locale/facet design?
Best, Vicente
It is... But when ICU does not provide immutable collator object and cloning collator object is too damn heavy such that it become terribly slow then I fallback to TLS and mutate TLS installed objects. Which allows be to give reasonable performance while keeping it invisible for user and fully thread safe. For non ICU based facets, there is no such problem. Artyom Beilis -------------- CppCMS - C++ Web Framework: http://cppcms.sf.net/ CppDB - C++ SQL Connectivity: http://cppcms.sf.net/sql/cppdb/
Artyom Beilis wrote:
From: Vicente Botet <vicente.botet@>
For example xalloc is thread safe in both MSVC and GCC libraries.
I don't like this kind of design, which forces the user to initialize the library. I will expect we can do better.
Global constructor?
I don't see what you mean.
I mean that all static ids are installed in global constructor like:
src/foo.cpp
int get_id() { static int v = std::ios_base::xalloc(); return v; }
namespace details { struct installer { installer() { get_id(); } } instance; }
This makes sure that if somebody links with the library the static ids (facet's ids) and static variables allocated with xalloc are installed.
Oh, I see. Unfortunately this technique can not be applied when the library is header-only, isn't it?
Of course this would not solve problem when two threads try dynamically load the same library linked with this one.
But this is quite extreme case.
And singleton can't be done in header only library under dll platform as well.
Isn't Boost.Flyweight a header only library?
I'm not familiar with it. So I don't know.
Its is quite heavy, but it seems to be working.
So... It is problem. However it is not problem for standard submission as you always have library - standard c++ library which by no means can be header only.
Well for the standard submission there is thread or module issue if this state is included directly on the ios state and the default facet is already imbue with the associated facets ;-). Of course this is something I cannot do with Boost.Chorno, or I don't know how to do it :'(.
But you can fallback to default if facet is not installed, i.e.:
if(!have_facet<foo>(loc)) // Do something as std::locale::classic dooes else use_facet<foo>(loc).bar(...)
Same for configuration. Ugly but useful for high level functions.
Yes, this is a good technique so that imbuing is not done by the library, only by the user. However this technique doesn't solves the multi-threaded imbuing issue.
Now ios state flags should not be thread safe as they stream specific and I don't expect from user to alter iostream flags from different threads.
(*) With ICU not all facets are immutable so in some places I use TLS to improve performance as for example with collator.
Why do you need mutable facets? Isn't this against the locale/facet design?
Best, Vicente
It is... But when ICU does not provide immutable collator object and cloning collator object is too damn heavy such that it become terribly slow then I fallback to TLS and mutate TLS installed objects. Which allows be to give reasonable performance while keeping it invisible for user and fully thread safe.
For non ICU based facets, there is no such problem.
I guess we can define always immutable facets that reference a mutable storage. Facet should be lightweight so that the creation doesn't takes too much time. Thanks for all your comments, very instructive. Best, Vicente -- View this message in context: http://boost.2283326.n4.nabble.com/iostreams-io-state-savers-Request-for-int... Sent from the Boost - Dev mailing list archive at Nabble.com.
--- On Thu, 11/10/11, Vicente Botet <vicente.botet@wanadoo.fr> wrote:
This makes sure that if somebody links with the
From: Vicente Botet <vicente.botet@wanadoo.fr> library the
static ids (facet's ids) and static variables allocated with xalloc are installed.
Oh, I see. Unfortunately this technique can not be applied when the library is header-only, isn't it?
Not... but you can use anonymous namespace and it would work but it is little big ugly.
But you can fallback to default if facet is not
installed, i.e.:
if(!have_facet<foo>(loc)) // Do something as std::locale::classic
dooes
else use_facet<foo>(loc).bar(...)
Same for configuration. Ugly but useful for high level functions.
Yes, this is a good technique so that imbuing is not done by the library, only by the user. However this technique doesn't solves the multi-threaded imbuing issue.
But I don't expect from user to change locale in a stream and use it in same time?! In any case you can do: try { foo const &f =use_facet<foo>(f); // do required } catch(std::bad_cast const &) { // do default }
For non ICU based facets, there is no such problem.
I guess we can define always immutable facets that reference a mutable storage. Facet should be lightweight so that the creation doesn't takes too much time.
No actually, it is not. facets are not generated frequently. locale is cheap to copy because facets are reference counted so you create facet once and use it for long time. For example messages facet can load dictionaries convert their encoding, it is by no means cheap. But you create it once and then reuse it. Artyom Beilis -------------- CppCMS - C++ Web Framework: http://cppcms.sf.net/ CppDB - C++ SQL Connectivity: http://cppcms.sf.net/sql/cppdb/
--- On Thu, 11/10/11, Vicente Botet<vicente.botet@wanadoo.fr> wrote:
This makes sure that if somebody links with the
From: Vicente Botet<vicente.botet@wanadoo.fr> library the
static ids (facet's ids) and static variables allocated with xalloc are installed.
Oh, I see. Unfortunately this technique can not be applied when the library is header-only, isn't it?
Not... but you can use anonymous namespace and it would work but it is little big ugly.
But you can fallback to default if facet is not installed, i.e.: if(!have_facet<foo>(loc)) // Do something as std::locale::classic dooes else use_facet<foo>(loc).bar(...)
Same for configuration. Ugly but useful for high level functions. Yes, this is a good technique so that imbuing is not done by the library, only by the user. However this technique doesn't solves the multi-threaded imbuing issue. But I don't expect from user to change locale in a stream and use it in same time?!
In any case you can do:
try { foo const&f =use_facet<foo>(f); // do required } catch(std::bad_cast const&) { // do default } I'm missing surely something. Recall, I'm a beginner. How do you ensure
Le 10/11/11 20:44, Artyom Beilis a écrit : that any stream has associated a locale that contains your facet?
For non ICU based facets, there is no such problem.
I guess we can define always immutable facets that reference a mutable storage. Facet should be lightweight so that the creation doesn't takes too much time. No actually, it is not. facets are not generated frequently. locale is cheap to copy because facets are reference counted so you create facet once and use it for long time. For example messages facet can load dictionaries convert their encoding, it is by no means cheap. But you create it once and then reuse it.
How do you reach to reuse them. For example if we use a string stream we will need to add the facet. I guess that this operation will not take too much time, and in any case I expect it to load a dictionary that has already been loaded. So I suspect that the facet should only contain a key identifying the dictionary or a reference to it. Vicente
From: Vicente J. Botet Escriba <vicente.botet@wanadoo.fr>
try { foo const&f =use_facet<foo>(f); // do required } catch(std::bad_cast const&) { // do default }
I'm missing surely something. Recall, I'm a beginner. How do you ensure that any stream has associated a locale that contains your facet?
You don't. For example std::ostream &operator<<(std::ostream &out,my_time const &tm) { std::locale l = out.getloc(); if(std::has_facet<my_time_facet>(loc)) { my_time_facet const &f=std::use_facet<my_time_facet>(loc); f.print_time(tm,out); } else { // user had not installed anything interesting so fallback for something. my_time::print_iso_time(tm,out); } return out; } It is not YOUR job to install facets. Ideally. std::locale should include "Classic" version of my_time_facet. But without it you can do what I had shown above.
No actually, it is not. facets are not generated frequently. locale is cheap to copy because facets are reference counted so you create facet once and use it for long time. For example messages facet can load dictionaries convert their encoding, it is by no means cheap. But you create it once and then reuse it.
How do you reach to reuse them.
For example if we use a string stream we will need to add the facet.
No, you don't. You don't add facet unless you need to bring some locale specific data.
I guess that this operation will not take too much time, and in any case I expect it to load a dictionary that has already been loaded. So I suspect that the facet should only contain a key identifying the dictionary or a reference to it.
Vicente
The process works this way (with Boost.Locale and even with std::locale. Boost Locale. ------------ std::locale l=generator(""); // load dictionaries creates facets etc std::cout.imbue(l); l << as::time << time(0); With std::locale ------------------ std::locale l(""); // load dictionaries create facets etc. std::cout.imbue(l); std::use_facet<std::time_put>(std::cout.getloc()).put(std::cout,std::cout,' ',localtime(time),'c',0); All the data created when you create locale object. You don't alter it afterwards. Now as C++ standard provided by default all facets even if l is C locale it will have facet you need. But from user point of view you can fallback to something if custom locale facet is not installed. I hope it is clear or we talk about different things Artyom
Le 10/11/11 23:18, Artyom Beilis a écrit :
From: Vicente J. Botet Escriba<vicente.botet@wanadoo.fr>
try { foo const&f =use_facet<foo>(f); // do required } catch(std::bad_cast const&) { // do default }
I'm missing surely something. Recall, I'm a beginner. How do you ensure that any stream has associated a locale that contains your facet? You don't. For example
std::ostream&operator<<(std::ostream&out,my_time const&tm) { std::locale l = out.getloc(); if(std::has_facet<my_time_facet>(loc)) { my_time_facet const&f=std::use_facet<my_time_facet>(loc); f.print_time(tm,out); } else { // user had not installed anything interesting so fallback for something. my_time::print_iso_time(tm,out); } return out; }
It is not YOUR job to install facets.
Ideally. std::locale should include "Classic" version of my_time_facet. But without it you can do what I had shown above.
No actually, it is not. facets are not generated frequently. locale is cheap to copy because facets are reference counted so you create facet once and use it for long time. For example messages facet can load dictionaries convert their encoding, it is by no means cheap. But you create it once and then reuse it.
How do you reach to reuse them.
For example if we use a string stream we will need to add the facet. No, you don't.
You don't add facet unless you need to bring some locale specific data.
I guess that this operation will not take too much time, and in any case I expect it to load a dictionary that has already been loaded. So I suspect that the facet should only contain a key identifying the dictionary or a reference to it.
Vicente
The process works this way (with Boost.Locale and even with std::locale.
Boost Locale. ------------
std::locale l=generator(""); // load dictionaries creates facets etc std::cout.imbue(l); l<< as::time<< time(0);
With std::locale ------------------ std::locale l(""); // load dictionaries create facets etc.
std::cout.imbue(l);
std::use_facet<std::time_put>(std::cout.getloc()).put(std::cout,std::cout,' ',localtime(time),'c',0);
All the data created when you create locale object. You don't alter it afterwards.
Now as C++ standard provided by default all facets even if l is C locale it will have facet you need. But from user point of view you can fallback to something if custom locale facet is not installed.
I hope it is clear or we talk about different things
Your last example is exactly the context to which I was referring to. The user needs to install some specific facets that depend on the locale, as the standard locale doesn't do it for you. So the question is. How can you associate a specific facet when you change the locale of a stream. I know that we can register callbacks for a specific ios_base, but what I think is needed is to register a callback for any ios_base object. Another way to see this problem is as follows. As you said , when there is no specific facet associated to the locale of a ios_base, no need to instal one, just take a default behavior. It would be great if the default behavior could be configured either depending on the locale name or as a global default facet that could be set by the user. I don't know if my concerns are more clear now. Thanks again, Vicente
----- Original Message -----
From: Vicente J. Botet Escriba <vicente.botet@wanadoo.fr>
Another way to see this problem is as follows. As you said , when there is no specific facet associated to the locale of a ios_base, no need to instal one, just take a default behavior. It would be great if the default behavior could be configured either depending on the locale name or as a global default facet that could be set by the user.
I don't know if my concerns are more clear now.
Thanks again, Vicente
I see. There are two ways to get information about current facet: 1. Using Boost.Locale's boost::locale::info facet and you can get the ISO-639-1 language code 2. Using std::locale::name() member function. Now about the second. It has two problems: 1. Names other then "C" and "POSIX" are not standardized 2. There are two major names around: a) POSIX names (used by GCC on Linux) like en_US.UTF-8 - you can parse it and get the ISO-639 language code. b) Windows names (used by MSVC) like Englush_USA.1252 you get windows specific language name that can be converted to ISO code you just need to create a conversion table. If your data is relatively small few K you can use it and behave according to it. This is actually what ICU does - locale name is just name and when you create some formatting object it look ups for the resources. It is heavier and wasteful but works. Artyom Beilis -------------- CppCMS - C++ Web Framework: http://cppcms.sf.net/ CppDB - C++ SQL Connectivity: http://cppcms.sf.net/sql/cppdb/
Le 12/11/11 12:09, Artyom Beilis a écrit :
----- Original Message -----
From: Vicente J. Botet Escriba<vicente.botet@wanadoo.fr>
Another way to see this problem is as follows. As you said , when there is no specific facet associated to the locale of a ios_base, no need to instal one, just take a default behavior. It would be great if the default behavior could be configured either depending on the locale name or as a global default facet that could be set by the user.
I don't know if my concerns are more clear now.
Thanks again, Vicente
I see.
There are two ways to get information about current facet:
1. Using Boost.Locale's boost::locale::info facet and you can get the ISO-639-1 language code 2. Using std::locale::name() member function.
Now about the second. It has two problems:
1. Names other then "C" and "POSIX" are not standardized
2. There are two major names around: a) POSIX names (used by GCC on Linux) like en_US.UTF-8 - you can parse it and get the ISO-639 language code. b) Windows names (used by MSVC) like Englush_USA.1252 you get windows specific language name that can be converted to ISO code you just need to create a conversion table.
If your data is relatively small few K you can use it and behave according to
it. This is actually what ICU does - locale name is just name and when you create some formatting object it look ups for the resources. It is heavier and wasteful but works.
OK I see. I will take a look at boost::locale::info facet to see if I can use it internally as index of the default chrono facets behavior. You have convinced me about abandoning the use of whatever messages facet. Thanks, Vicente
participants (3)
-
Artyom Beilis -
Vicente Botet -
Vicente J. Botet Escriba