I've spent a tiny amount of time looking over the flyweight documentation and some code and I have a couple minor observations along with a really dumb question. a) when I see boost::flyweight::flyweights it confuses me. I suppose its no big deal. b) I'm curious about the serialization implementation. Why didn't you choose just to serialize the factory? This seems to me the most obvious implementation and I wonder why you didn't feel it was a good choice. template< class Archive, typename T,typename Arg1,typename Arg2,typename Arg3
void serialize( Archive& ar,const ::boost::flyweights::flyweight<T,Arg1,Arg2,Arg3>& f, const unsigned int version) { typedef ::boost::flyweights::flyweight<T,Arg1,Arg2,Arg3> flyweight; ar & flyweight::factory(); // serialization tracking guarentees only one copy ar & f.h; // serialization tracking optimizes down to an integer object id } This might not be a good idea if flyweight reference count isn't used. So another idea might be: template< class Archive, typename T,typename Arg1,typename Arg2,typename Arg3
void save( Archive& ar,const ::boost::flyweights::flyweight<T,Arg1,Arg2,Arg3>& f, const unsigned int version) { typedef ::boost::flyweights::flyweight<T,Arg1,Arg2,Arg3> flyweight; ar << f.h; // serialization tracking optimizes down to an integer object id } template< class Archive, typename T,typename Arg1,typename Arg2,typename Arg3
void load( Archive& ar,const ::boost::flyweights::flyweight<T,Arg1,Arg2,Arg3>& f, const unsigned int version) { typedef ::boost::flyweights::flyweight<T,Arg1,Arg2,Arg3> flyweight; T t; ar >> t; // serialization tracking optimizes down to an integer object id f.factory.insert(t); // or something along those lines //fix up object address of t - off hand I don't remember the syntax } In short, I don't see why a "helper" is required. Robert Ramey
Hello Robert, ----- Mensaje original ----- De: Robert Ramey <ramey@rrsd.com> Fecha: Domingo, Febrero 3, 2008 8:01 pm Asunto: [boost] question about flyweight Para: boost@lists.boost.org
I've spent a tiny amount of time looking over the flyweight documentation and some code and I have a couple minor observations along with a really dumb question.
a) when I see boost::flyweight::flyweights it confuses me. I suppose its no big deal.
It's boost::flyweights::flyweight, actually. The reason for the spurious ess is explained at: http://lists.boost.org/Archives/boost/2008/01/132628.php
b) I'm curious about the serialization implementation. Why didn't you choose just to serialize the factory? This seems to me the most obvious implementation and I wonder why you didn't feel it was a good choice. [...] In short, I don't see why a "helper" is required.
Details aside, it all boils down to this: ar << f.h; This only works off the shelf if f.h is a pointer, which is not the case in general (the type of this handle is provided by the particular factory used, and need not be a pointer, for instance in set_factroy is a std::set::iterator). Briefly put, the serialization helper used by flyweight mimics for handle_type the tracking facilities Boost.Serialization provides natively for pointers. Joaquín M López Muñoz Telefónica, Investigación y Desarrollo
"JOAQUIN LOPEZ MU?Z" wrote:
Hello Robert,
----- Mensaje original ----- De: Robert Ramey <ramey@rrsd.com> Fecha: Domingo, Febrero 3, 2008 8:01 pm Asunto: [boost] question about flyweight Para: boost@lists.boost.org
I've spent a tiny amount of time looking over the flyweight documentation and some code and I have a couple minor observations along with a really dumb question.
a) when I see boost::flyweight::flyweights it confuses me. I suppose its no big deal.
It's boost::flyweights::flyweight, actually. The reason for the spurious ess is explained at:
http://lists.boost.org/Archives/boost/2008/01/132628.php
b) I'm curious about the serialization implementation. Why didn't you choose just to serialize the factory? This seems to me the most obvious implementation and I wonder why you didn't feel it was a good choice. [...] In short, I don't see why a "helper" is required.
Details aside, it all boils down to this:
ar << f.h;
This only works off the shelf if f.h is a pointer, which is not the case in general (the type of this handle is provided by the particular factory used, and need not be a pointer, for instance in set_factroy is a std::set::iterator). Briefly put, the serialization helper used by flyweight mimics for handle_type the tracking facilities Boost.Serialization provides natively for pointers.
The default is to track if and only if its serialized through a pointer. If the tracking level serialization trait for flyweight<T> is set to track_always it should work with all objects. I would think that you could get the effect you intend by just setting this serialization trait.for these types. Robert Ramey
Joaquín M López Muñoz Telefónica, Investigación y Desarrollo _______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Robert Ramey ha escrito:
"JOAQUIN LOPEZ MU?Z" wrote:
Hello Robert,
----- Mensaje original ----- De: Robert Ramey <ramey@rrsd.com> Fecha: Domingo, Febrero 3, 2008 8:01 pm Asunto: [boost] question about flyweight Para: boost@lists.boost.org
[...]
b) I'm curious about the serialization implementation. Why didn't you choose just to serialize the factory? This seems to me the most obvious implementation and I wonder why you didn't feel it was a good choice. [...] In short, I don't see why a "helper" is required.
Details aside, it all boils down to this:
ar << f.h;
This only works off the shelf if f.h is a pointer, which is not the case in general (the type of this handle is provided by the particular factory used, and need not be a pointer, for instance in set_factroy is a std::set::iterator). Briefly put, the serialization helper used by flyweight mimics for handle_type the tracking facilities Boost.Serialization provides natively for pointers.
The default is to track if and only if its serialized through a pointer.
If the tracking level serialization trait for flyweight<T> is set to track_always it should work with all objects.
I would think that you could get the effect you intend by just setting this serialization trait.for these types.
I can do something of sorts, but it's not as efficient as with the aid of a helper: Suppose T is tracked, then your proposed approach would be something like this: template< class Archive, typename T,typename Arg1,typename Arg2,typename Arg3
void save( Archive& ar,const ::boost::flyweights::flyweight<T,Arg1,Arg2,Arg3>& f, const unsigned int version) { typedef ::boost::flyweights::flyweight<T,Arg1,Arg2,Arg3> flyweight; ar << &f.get(); } template< class Archive, typename T,typename Arg1,typename Arg2,typename Arg3
void load( Archive& ar,const ::boost::flyweights::flyweight<T,Arg1,Arg2,Arg3>& f, const unsigned int version) { typedef ::boost::flyweights::flyweight<T,Arg1,Arg2,Arg3> flyweight; T* p; ar >> p; f=*p; // HERE if(&f.get()!=p){ ar.reset_object_address(&f.get(),p); delete p; } } The problem is with the line marked //HERE : we are incurring a lookup operation for each duplicate flyweight, while this is avoided when a helper is used. As a matter of fact, I measured this approach against the helper implementation and the performance of the latter is easily 2x or 3x better (I don't remember the exact figures, I can retest if you're interested.) Joaquín M López Muñoz Telefónica, Investigación y Desarrollo
template< class Archive, typename T,typename Arg1,typename Arg2,typename Arg3
void save( Archive& ar,const ::boost::flyweights::flyweight<T,Arg1,Arg2,Arg3>& f, const unsigned int version) { typedef ::boost::flyweights::flyweight<T,Arg1,Arg2,Arg3> flyweight; ar << f.get(); // use serialization tracking - no need to call helper } template< class Archive, typename T,typename Arg1,typename Arg2,typename Arg3
void load( Archive& ar,const ::boost::flyweights::flyweight<T,Arg1,Arg2,Arg3>& f, const unsigned int version) { typedef ::boost::flyweights::flyweight<T,Arg1,Arg2,Arg3> flyweight; T t; ar >> t; f = t; // look up here ar.reset_object_address(& f.get(), &t); } vs. // from flyweight serialize template< class Archive, typename T,typename Arg1,typename Arg2,typename Arg3
void load( Archive& ar,::boost::flyweights::flyweight<T,Arg1,Arg2,Arg3>& f, const unsigned int version ){ typedef ::boost::flyweights::flyweight<T,Arg1,Arg2,Arg3> flyweight; typedef ::boost::flyweights::detail::serialization_helper<flyweight> helper; typedef typename helper::size_type size_type; helper& hlp= ::boost::flyweights::detail::get_serialization_helper<flyweight>(ar); size_type n=0; ar>>make_nvp("item",n); if(n>hlp.size()){ throw_exception( archive::archive_exception(archive::archive_exception::other_exception)); } else if(n==hlp.size()){ ::boost::flyweights::detail::archive_constructed<T> v("value",ar,version); hlp.push_back(flyweight(v.get())); // Doesn't push_back need to do a find? } f=hlp[n]; } I've delved into the helper. I kind of ran out of gas at multi-index. Doesn't this require a lookup to maintain the mult-indexed table?. Also using the helper entails a lot more generated code. It looks like a bunch more code generated for each type flyweight. I'm not convinced that there is a real saving here. If you dropped the helper, then you're class would be serializable right now and it would entail a lot less code. You could always make it more elaborate later. Robert Ramey
Robert Ramey ha escrito: [...]
template< class Archive, typename T,typename Arg1,typename Arg2,typename Arg3
void load( Archive& ar,const ::boost::flyweights::flyweight<T,Arg1,Arg2,Arg3>& f, const unsigned int version) { typedef ::boost::flyweights::flyweight<T,Arg1,Arg2,Arg3> flyweight; T t; ar >> t; f = t; // look up here ar.reset_object_address(& f.get(), &t); }
vs.
// from flyweight serialize
template< class Archive, typename T,typename Arg1,typename Arg2,typename Arg3
void load( Archive& ar,::boost::flyweights::flyweight<T,Arg1,Arg2,Arg3>& f, const unsigned int version ){ typedef ::boost::flyweights::flyweight<T,Arg1,Arg2,Arg3> flyweight; typedef ::boost::flyweights::detail::serialization_helper<flyweight> helper; typedef typename helper::size_type size_type; helper& hlp= ::boost::flyweights::detail::get_serialization_helper<flyweight>(ar); size_type n=0; ar>>make_nvp("item",n); if(n>hlp.size()){ throw_exception( archive::archive_exception(archive::archive_exception::other_exception)); } else if(n==hlp.size()){ ::boost::flyweights::detail::archive_constructed<T> v("value",ar,version); hlp.push_back(flyweight(v.get())); // Doesn't push_back need to do a find? } f=hlp[n]; }
I've delved into the helper. I kind of ran out of gas at multi-index.
:-) I know my code gets cryptic sometimes. Sorry for that. The helper is just a table of flyweight objects with the following access methods: 1. Given a flyweight f, it returns the position n of an equivalent object stored in the table (if there is any). 2. Given a position n, it returns the flyweight at that position. It's a kind of vector of flyweights with hash-based lookup.
Doesn't this require a lookup to maintain the mult-indexed table?
Well, it turns push_back does indeed induces an insertion into the hashed index, but this is an oversight from my part: we could do with a singly indexed container and no lookup at all during the load phase, we need only use different helpers for loading and saving. Either way, the actual measurements showed the helper version outperformed the version based on T tracking, even with this unnecessary lookup in place. Note that the non-helper version has an extra lookup lurking inside reset_object_address. For your reference, the helper-based loading algorithm reduces to: 1. Load position n (constant time) 2. If the table has size>=n, get flyweight at position n and assign to our loading object (constant time) 3. Otherwise, load a T object, assign to our loading object (lookup needed) and store a copy on the table at position n (constant time). It's hard to beat this up, it's as simple as it can get. In particular, when loading a duplicate flyweight the executed code is extremely lean.
Also using the helper entails a lot more generated code. It looks like a bunch more code generated for each type flyweight. I'm not convinced that there is a real saving here.
I've got nothing to say about the extra generated code except that I don't think this is so important (others might differ), but there is definitely a high impact on performance. It's not a theoretical analysis, I measured it.
If you dropped the helper, then you're class would be serializable right now and it would entail a lot less code. You could always make it more elaborate later.
If I released the library with serialization algorithm A and later on I migrated to algorithm B, I'd have to maintain some legacy code to support old A-archives. This is why I'd rather get it right at first shot. IMHO we should shift our discussion focus to whether helpers make sense or not in the first place, regardless of their particular applications. It's only on its general merits that helpers should be included or not, of course. My particular application at least shows they provide for much better performance. Joaquín M López Muñoz Telefónica, Investigación y Desarrollo
participants (3)
-
"JOAQUIN LOPEZ MU?Z" -
Joaquín Mª López Muñoz
-
Robert Ramey