Formal review request: Flyweight
Hello, I request that the Candidate Boost Flyweight Library be accepted into the review queue. The latest public version of the library is described at http://lists.boost.org/Archives/boost/2007/11/130274.php There is no pre-appointed review manager for this library. Thank you, Joaquín M López Muñoz Telefónica, Investigación y Desarrollo
I just read a few pages of the documentation and I think I understand the problem this library addresses. Since the library is designed to be like a drop-in replacement for immutable shared objects, there are similarities with shared_ptr: flyweight<T> is designed to be a drop-in replacement for T const, whereas shared_ptr<T> is a drop-in replacement for T *. Even issues related to serialization and factories are similar between shared_ptr<T> and flyweight<T>. I am pointing out the similarities because the flyweight framework lacks a very important property of shared_ptr, which I think is directly applicable, and it has to do with the way flyweight<T> is configured. This is how two flyweight<string> objects can be configured to use different pools: struct name_tag1{}; typedef flyweight<std::string,tag<name_tag1> > str1_t; struct name_tag2{}; typedef flyweight<std::string,tag<name_tag2> > str2_t; The problem is that now str1_t and str2_t are distinct types in the type system, yet they are identical semantically. Suppose I have a function that takes a flyweight string like this: void foo( flyweight<std::string> const & s ); Even if implicit conversions between different configurations of flyweight<std::string> exists, calling foo with str1_t or str2_t would create unwanted temporary objects. Besides, such conversions will create a default-configured flyweight<std::string>, which is not desirable. The only option I would have is to make foo a template which is unfortunate because it would increase physical coupling unnecessarily. Compare this with the way users supply custom allocators to shared_ptr: http://www.boost.org/libs/smart_ptr/shared_ptr.htm#allocator_constructor Two shared_ptr<T> objects that use different allocators, once initialized, are indistinguishable from each other because they are of the same type, shared_ptr<T>. I would like flyweight<T> configuration to work similarly. -- Emil Dotchevski Reverge Studios, Inc. http://www.revergestudios.com/reblog/index.php?n=ReCode
Hi Emil ----- Mensaje original ----- De: Emil Dotchevski <emil@revergestudios.com> Fecha: Miércoles, Noviembre 14, 2007 8:36 pm Asunto: Re: [boost] Formal review request: Flyweight Para: boost@lists.boost.org [...]
struct name_tag1{}; typedef flyweight<std::string,tag<name_tag1> > str1_t;
struct name_tag2{}; typedef flyweight<std::string,tag<name_tag2> > str2_t;
The problem is that now str1_t and str2_t are distinct types in the type system, yet they are identical semantically. Suppose I have a function that takes a flyweight string like this:
void foo( flyweight<std::string> const & s );
Even if implicit conversions between different configurations of flyweight<std::string> exists, calling foo with str1_t or str2_t would create unwanted temporary objects. Besides, such conversions will create a default-configured flyweight<std::string>, which is not desirable. The only option I would have is to make foo a template which is unfortunate because it would increase physical coupling unnecessarily.
Compare this with the way users supply custom allocators to shared_ptr:
http://www.boost.org/libs/smart_ptr/shared_ptr.htm#allocator_constructo r
Two shared_ptr<T> objects that use different allocators, once initialized, are indistinguishable from each other because they are of the same type, shared_ptr<T>.
I would like flyweight<T> configuration to work similarly.
Your proposal is, as I see it, to some extent akin to a *local_flyweight* variant Tobias Schwinger and I were privately discussing some time ago: The shared_ptr allocator ctor accepts the *context* the shared_ptr object needs during its lifetime, namely the associated allocator and deleter. For flyweight, the context consists of the flyweight factory and some synchronization objects. So, the analog of shared_ptr allocator ctor in the realm of flyweight would be something like: template<typename T1,...,typename Tn> local_flyweight([const] T1&,...,[const] Tn&,context *); where context is an associated type containing all the static stuff, the factory and such. This can be done and would achieve a separation of "domains" without using different types, just as you demand (if I'm getting you right). This local_flyweight<> has other features apart from domain separation, like for instance the ability for the user to control the lifetime of the flyweight and the possibility of creating unlimited domains at run time (tag separation is a compile time artifact and as such the number of different domains one can create this way is fixed at programming time.) Why implementing this in a different class than flyweight<> itself? The reason is the value objects of local_flyweight<> (the elements stored at the flyweight factory, which contain the shared data as well as some bookkeeping info) must have a pointer to its context, whereas in the case of flyweight<> this context is associated to the type itself, thus saving us a pointer per value. Would this local_flyweight<> address your needs? If so, I plan to evolve the idea over time and either have it implemented for the review or else included in the future work section. Let me know what you think about it. Joaquín M López Muñoz Telefónica, Investigación y Desarrollo
Why implementing this in a different class than flyweight<> itself? The reason is the value objects of local_flyweight<> (the elements stored at the flyweight factory, which contain the shared data as well as some bookkeeping info) must have a pointer to its context, whereas in the case of flyweight<> this context is associated to the type itself, thus saving us a pointer per value.
If we look at shared_ptr again, there are competing, policy-based smart pointer designs, but they all lack the ability to work with incomplete types. For anything to be a drop-in replacement for T *, it must work with incomplete types. In other words, a policy-based smart pointer design can be "smart", but it wouldn't be much of a "pointer". Similarly, if flyweight<std::string> aims to be a drop-in replacement for an immutable std::string, it should have the key properties of std::strings. For example: std::string s1; std::string s2; Now I want to make use of flyweight, so I write: flyweight<std::string,tag<tag_name> > s1; flyweight<std::string,tag<tag_ip> > s2; I started with two objects of the same type, and ended up with two objects of different types. As a user, I have two options: to get unwanted temporaries at various points in my code, or to templatize sections of my code to avoid them. Either choice would be a show-stopper for me. Also, shared_ptr needs proper factory support to compliment serialization of shared_ptr objects, which is the same stuff needed to implement flyweight. I think that removing the policies from flyweight will make it possible to implement the factories and the serialization support for shared_ptr directly, making the flyweight class template a thin wrapper over shared_ptr, and thus much lighter Boost library. -- Emil Dotchevski Reverge Studios, Inc. http://www.revergestudios.com/reblog/index.php?n=ReCode
Emil Dotchevski ha escrito: [...]
If we look at shared_ptr again, there are competing, policy-based smart pointer designs, but they all lack the ability to work with incomplete types. For anything to be a drop-in replacement for T *, it must work with incomplete types. In other words, a policy-based smart pointer design can be "smart", but it wouldn't be much of a "pointer".
Similarly, if flyweight<std::string> aims to be a drop-in replacement for an immutable std::string, it should have the key properties of std::strings. For example:
std::string s1; std::string s2;
Now I want to make use of flyweight, so I write:
flyweight<std::string,tag<tag_name> > s1; flyweight<std::string,tag<tag_ip> > s2;
I started with two objects of the same type, and ended up with two objects of different types. As a user, I have two options: to get unwanted temporaries at various points in my code, or to templatize sections of my code to avoid them.
Either choice would be a show-stopper for me.
As far as I can see, this is a re-statement of your original complaint. You can avoid proliferation of tagged flyweights by using the (non-existent yet) local_flyweight, which defers domain selection to run-time. Is this not a solution to your concern? Another possibility, and this might be more likeable to you, is to take advantage of the implicit conversion from flyweight<T> to const T&, and instead of having void foo( flyweight<std::string> const & s ); write void foo( std::string const & s ); which will work with any flyweight of std::string, regardless of its being tagged or not and the different configuration policies used. No temporaries are created. And without templatizing anything. Is this more in line with what you have in mind?
Also, shared_ptr needs proper factory support to compliment serialization of shared_ptr objects, which is the same stuff needed to implement flyweight.
I don't understand this. What factory support for shared_ptr are you referring to?
I think that removing the policies from flyweight will make it possible to implement the factories and the serialization support for shared_ptr directly, making the flyweight class template a thin wrapper over shared_ptr, and thus much lighter Boost library.
Again I don't get it. What is the relation between removing policies from flyweight<> and implementing serialization support for shared_ptr? Could you please elaborate your point? Thank you! Joaquín M López Muñoz Telefónica, Investigación y Desarrollo
Another possibility, and this might be more likeable to you, is to take advantage of the implicit conversion from flyweight<T> to const T&, and instead of having
void foo( flyweight<std::string> const & s );
write
void foo( std::string const & s );
If you have two systems that internally use std::string to store strings, they could conceivably communicate in terms of a char const * interface; but the fact that you can strip the std::string semantics, doesn't mean that you always want to. Similarly, yes, you can pass a flyweight<std::string> as an immutable std::string, but in doing so you're getting rid of the flyweight semantics. Sometimes this is exactly what you want, and sometimes it isn't. In the latter case you have the problem I outlined.
Also, shared_ptr needs proper factory support to compliment serialization of shared_ptr objects, which is the same stuff needed to implement flyweight.
I don't understand this. What factory support for shared_ptr are you referring to?
The one that does not exist (yet) :) Let's say you have two shared_ptr<foo> objects, pa and pb, that point to different foo objects. How do you serialize them? Consider that they may have been created by different factories and so at write time you need to save that information so that at read time you know which factory to use for pa and for pb. If I understand correctly what flyweight factories are, and how flyweight is serialized, it solves a very similar problem. So what I mean is, perhaps this problem can be solved independently of flyweight and applied to both flyweight and shared_ptr. Even better (in my opinion), the problem can be solved for shared_ptr, and then flyweight<T> could be implemented over shared_ptr<T>. -- Emil Dotchevski Reverge Studios, Inc. http://www.revergestudios.com/reblog/index.php?n=ReCode
Emil Dotchevski ha escrito:
Another possibility, and this might be more likeable to you, is to take advantage of the implicit conversion from flyweight<T> to const T&, and instead of having
void foo( flyweight<std::string> const & s );
write
void foo( std::string const & s );
If you have two systems that internally use std::string to store strings, they could conceivably communicate in terms of a char const * interface; but the fact that you can strip the std::string semantics, doesn't mean that you always want to.
Similarly, yes, you can pass a flyweight<std::string> as an immutable std::string, but in doing so you're getting rid of the flyweight semantics. Sometimes this is exactly what you want, and sometimes it isn't. In the latter case you have the problem I outlined.
I understand your point and I think it's respectable, but in the end your position is tantamount to banning policy-based classes, and with that I obviously cannot agree. You seem to object to using templates for one of the very purposes they were designed for: making the same code work for different types.
Also, shared_ptr needs proper factory support to compliment serialization of shared_ptr objects, which is the same stuff needed to implement flyweight.
I don't understand this. What factory support for shared_ptr are you referring to?
The one that does not exist (yet) :)
Let's say you have two shared_ptr<foo> objects, pa and pb, that point to different foo objects. How do you serialize them?
Consider that they may have been created by different factories and so at write time you need to save that information so that at read time you know which factory to use for pa and for pb. If I understand correctly what flyweight factories are, and how flyweight is serialized, it solves a very similar problem.
Well, I think flyweight<> does not do exactly what you think in this respect: there's no need to *find out* which flyweight factory a given flyweight object is associated to because any flyweight<...> type has a unique factory of its own, identifiable at compile-time. So, the info on what factory to use when (de)serializing a flyweight does not travel across the archive.
So what I mean is, perhaps this problem can be solved independently of flyweight and applied to both flyweight and shared_ptr. Even better (in my opinion), the problem can be solved for shared_ptr, and then flyweight<T> could be implemented over shared_ptr<T>.
Joaquín M López Muñoz Telefónica, Investigación y Desarrollo
I understand your point and I think it's respectable, but in the end your position is tantamount to banning policy-based classes, and with that I obviously cannot agree. You seem to object to using templates for one of the very purposes they were designed for: making the same code work for different types.
I was specifically addressing the design of the flyweight framework. There is significant overlap between the flyweight domain and the shared_ptr domain; in my opinion, policies are just as inappropriate for flyweight as for shared_ptr (or just as appropriate in both cases, depending on who you ask.) I'll make one last attempt... :) Let's say I have a vector< flyweight<string> > container. Let's also assume that, based on the value of the stored string object, I want to allocate the flyweight differently. No problem, right? All I have to do is write a custom factory, and change my container's value type: vector< flyweight<string,tag<my_factory> > >. So, my_factory inspects the string value passed to it, and uses the appropriate allocation strategy. But what if I can't hard-code the factory behavior? What if I want to allow user code to choose how each individual flyweight<string> is to be allocated? There are many operations on flyweight<string> that have nothing to do with how it was allocated, yet this design couples them physically with the factory.
Let's say you have two shared_ptr<foo> objects, pa and pb, that point to different foo objects. How do you serialize them?
Consider that they may have been created by different factories and so at write time you need to save that information so that at read time you know which factory to use for pa and for pb. If I understand correctly what flyweight factories are, and how flyweight is serialized, it solves a very similar problem.
Well, I think flyweight<> does not do exactly what you think in this respect
I realize that. But, *if* we come up with a "proper" serialization of shared_ptr (which is necessary anyway), and *if* the design of flyweight<> is altered like I am suggesting, then flyweight<> becomes (almost) trivial to implement. :) -- Emil Dotchevski Reverge Studios, Inc. http://www.revergestudios.com/reblog/index.php?n=ReCode
Emil Dotchevski a écrit :
I'll make one last attempt... :)
Let's say I have a vector< flyweight<string> > container. Let's also assume that, based on the value of the stored string object, I want to allocate the flyweight differently. No problem, right? All I have to do is write a custom factory, and change my container's value type: vector< flyweight<string,tag<my_factory> > >. So, my_factory inspects the string value passed to it, and uses the appropriate allocation strategy.
But what if I can't hard-code the factory behavior? What if I want to allow user code to choose how each individual flyweight<string> is to be allocated?
Why not make a factory that does dynamic dispatch?
Emil Dotchevski <emil <at> revergestudios.com> writes:
I'll make one last attempt... :)
Let's say I have a vector< flyweight<string> > container. Let's also assume that, based on the value of the stored string object, I want to allocate the flyweight differently. No problem, right? All I have to do is write a custom factory, and change my container's value type: vector< flyweight<string,tag<my_factory> > >. So, my_factory inspects the string value passed to it, and uses the appropriate allocation strategy.
But what if I can't hard-code the factory behavior? What if I want to allow user code to choose how each individual flyweight<string> is to be allocated?
There are many operations on flyweight<string> that have nothing to do with how it was allocated, yet this design couples them physically with the factory.
I have two observations regarding this: 1. I think the local_flyweight variation I presented a couple of posts ago addresses this use case. Do you see it otherwise? 2. Anyway, what do you want such a fine control over factories (no irony intended)? After all, having the flyweight value stored in one factory or another is a transparent aspect for most of flyweight<> use interface. The primary purpose of flyweight is to provide a replacement for const T that it's space efficient, no factories involved in this rationale. Think of the factory in flyweight<> as playing a similar role as the allocator in std::basic_string<>. Have you ever felt the need to have two std::string's using a different allocator object?
Let's say you have two shared_ptr<foo> objects, pa and pb, that point to different foo objects. How do you serialize them?
Consider that they may have been created by different factories and so at write time you need to save that information so that at read time you know which factory to use for pa and for pb. If I understand correctly what flyweight factories are, and how flyweight is serialized, it solves a very similar problem.
Well, I think flyweight<> does not do exactly what you think in this respect
I realize that. But, *if* we come up with a "proper" serialization of shared_ptr (which is necessary anyway), and *if* the design of flyweight<> is altered like I am suggesting, then flyweight<> becomes (almost) trivial to implement. :)
I haven't thought about how serialization of a potential local_flyweight<> would be, but it looks much more involved than pure flyweight<>, where the factory isn't actually serialized at all. Joaquín M López Muñoz Telefónica, Investigación y Desarrollo
Hi Joaquin, I have received your request and will add it to the review queue. Cheers, Ron On Nov 14, 2007, at 12:57 PM, Joaquín Mª López Muñoz wrote:
Hello,
I request that the Candidate Boost Flyweight Library be accepted into the review queue. The latest public version of the library is described at
http://lists.boost.org/Archives/boost/2007/11/130274.php
There is no pre-appointed review manager for this library.
Thank you,
Joaquín M López Muñoz Telefónica, Investigación y Desarrollo
_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/ listinfo.cgi/boost
On 11/14/07 11:57, Joaquín Mª López Muñoz wrote:
Hello,
I request that the Candidate Boost Flyweight Library be accepted into the review queue. The latest public version of the library is described at
Joaquin, I've only briefly looked at: http://svn.boost.org/svn/boost/sandbox/flyweight/libs/flyweight/doc/referenc... after being prompted by the post: http://lists.boost.org/Archives/boost/2007/12/131742.php containing: However it sounds like something that's probably implemented by (or at least easily implementable with) Joaquin M. Lopez' Flyweight library. However, I was a little worried that flyweight.html#flyweight contains: flyweight([const] T0& t0,[const] T1& t1); ... template<typename T0,...,typename Tn-1> flyweight([const] T0& t0,...,[const] Tn-1& tn-1); which looks like maybe there's some 'implementation overlap' between flyweight and the just approved functional/forward lib. I say 'implementation overlap' because it seem that both flyweight and forward must use some sort of BOOST_PREPROCESSOR macro's to generate the overloaded call arglist. If so, then it would be nice (for those trying to understand the implementation) if the forward and flyweight could reuse some common subset of these BOOST_PREPROCESSOR macros. IOW, I'm suggesting that the macros be refactored into some common library (probably the functional/forward library since that's essentially what's happening). Of course this suggestion is just based, as I said, on a brief look at your docs and maybe my jumping to a wrong conclusion. -regards, Larry
----- Mensaje original ----- De: Larry Evans <cppljevans@suddenlink.net> Fecha: Miércoles, Diciembre 19, 2007 8:47 pm Asunto: Re: [boost] Formal review request: Flyweight Para: boost@lists.boost.org
On 11/14/07 11:57, Joaquín Mª López Muñoz wrote:
Hello,
I request that the Candidate Boost Flyweight Library be accepted into> the review queue. The latest public version of the library is described at
Joaquin,
I've only briefly looked at:
http://svn.boost.org/svn/boost/sandbox/flyweight/ libs/flyweight/doc/reference/flyweight.html#flyweight [...] However, I was a little worried that flyweight.html#flyweight contains: flyweight([const] T0& t0,[const] T1& t1); ... template<typename T0,...,typename Tn-1> flyweight([const] T0& t0,...,[const] Tn-1& tn-1);
which looks like maybe there's some 'implementation overlap' between flyweight and the just approved functional/forward lib. I say 'implementation overlap' because it seem that both flyweight and forward must use some sort of BOOST_PREPROCESSOR macro's to generate the overloaded call arglist. If so, then it would be nice (for those trying to understand the implementation) if the forward and flyweight could reuse some common subset of these BOOST_PREPROCESSOR macros. IOW, I'm suggesting that the macros be refactored into some common library (probably the functional/forward library since that's essentially what's happening).
Of course this suggestion is just based, as I said, on a brief look at your docs and maybe my jumping to a wrong conclusion.
Yep, I agree with you my preprocessor code should be dropped in favor of functional/forward (+ functional/factory, if accepted), and I plan to do so in case my lib is accepted in Boost. For the moment being I prefer to leave the review version as is just as to not make it depedent on (currently) external libraries. Thank you for your suggestion, hope to see at the review, Joaquín M López Muñoz Telefónica, Investigación y Desarrollo
participants (7)
-
"JOAQUIN LOPEZ MU?Z" -
Emil Dotchevski -
Joaquin M Lopez Munoz
-
Joaquín Mª López Muñoz
-
Larry Evans -
Mathias Gaunard -
Ronald Garcia