[serialization] hacking serialization : handles to pointers
Ok I've spent a good chunk of my day making a set of custom archive class,
based on the portable_binary_iarchive and portable_binary_oarchive.
I have a non-virtual class hierarchy named Asset, in memory this is a cyclic
graph, but my requirements are that this structure cannot be serialized all
at once, it has to be broken up at boundaries. So the archiver serializes
the first Asset* (which is always the root object passed into operator&).
This object serializes as normally as per the boost serialization code,
except except that when subsequent Asset*'s are encountered I send the
pointer into an AssetManager class and translate the Asset* into a
ResourceID, and serialize the ResourceID instead of the object. This is
done by directly calling operator& inside save_override.
So far all of this works:
The reverse case of loading, by hooking load_override almost works. Almost,
because I have to apply a hack of skipping 4 bytes inside load_override,
before trying to read the ResourceID structure with operator&. I can't
figure out how to make it 'proper', without simply hard coding the 4 byte
read. As far as I can tell before save_override some bookkeeping
information is written, but the reverse operation in load_override has not
yet stripped this bookkeeping information from the stream, so these
functions aren't quite mirror operations. As best as I can tell the
function guilty of reading the bookkeeping information is load_pointer.
What I need is another function in the boost archive API that pulls the
bookkeeping information out of the stream safely, that I can call from
within load_override, since I have no intention of calling load_pointer
through any means.
Aside from the hack, it technically works, but even then I still have a few
aching problems:
I also have to hard-code the full list of derived Asset types and manually
provide specializations for all of them in save_override and load_override.
If I use the base class, I end up slicing my class down to its base, and
cannot serialize it. I can't make serialize virtual, since the intrusive
serialize methods are templates, but it certainly would solve the problem if
it were possible. In addition the bodies of all of my overrides are
completely identical except for the classname (AssetModel, AssetTexture,
etc). Which means I'll be wrapping the bodies of a generic save_override
and load_override in a macro, and have to manually add all Asset derived
classes to a list of classes inside my archive class. Which means that my
archiver cannot be generic, even though I have managed to make it a template
in the sense that the passed in asset manager and base asset types are
template parameters.
the type ResourceID is a std::string
saving...
class asset_oarchive : ...
void save_override(const AssetModel& t, int i)
{
if (!RootAssetSaved)
{
RootAssetSaved = true;
binary_oarchive_impl::save_override(t, i);
}
else
{
AssetManagerType::ResourceID ResourceId =
TheAssetManager->ResourceIdForAsset(&t);
operator&(ResourceId);
}
}
loading ...
class asset_iarchive : ...
void load_override(AssetModel*& t, int i)
{
if (!RootAssetLoaded)
{
RootAssetLoaded = true;
binary_iarchive_impl::load_override(t, i);
}
else
{
unsigned char c[10];
for (INT x=0; x<10; ++x)
{
c[x] = 0xFF;
}
for (INT x=0; x<4; ++x)
{
operator&(c[x]); //hackhack
}
AssetManagerType::ResourceID ResourceId;
operator&(ResourceId);
AssetType* a = TheAssetManager->FindAsset(ResourceId);
if (!a)
{
a = TheAssetManager->LoadAsset(ResourceId);
}
// TODO: throw on missing resources, potentially query asset
manager for substitutes
t = static_cast
Sean Cavanaugh wrote:
Ok I've spent a good chunk of my day making a set of custom archive class, based on the portable_binary_iarchive and portable_binary_oarchive.
I have a non-virtual class hierarchy named Asset, in memory this is a cyclic graph, but my requirements are that this structure cannot be serialized all at once, it has to be broken up at boundaries. So the archiver serializes the first Asset* (which is always the root object passed into operator&). This object serializes as normally as per the boost serialization code, except except that when subsequent Asset*'s are encountered
?which point to a previously serialized object? or any subsequent Asset *?
I send the pointer into an AssetManager class and translate the Asset* into a ResourceID, and serialize the ResourceID instead of the object. This is done by directly calling operator& inside save_override.
******* Hmmm this sounds to me exactly equivalent to what the serialization system does by default for tracked objects. Objects serialized through pointers are tracked by default. Your "ResourceID" seems a re-implemenation of the "object id" used by the serialization libary to track
...
I also have to hard-code the full list of derived Asset types and manually provide specializations for all of them in save_override and load_override.
**** Well, since they're different - I would expect each of them to have a different serialize function. If all the serialize functions are the same, it would seem that something should be moved from the derived class to the base class.
If I use the base class, I end up slicing my class down to its base, and cannot serialize it.
**** serializing through a base class pointer solves this problem as well.
I can't make serialize virtual, since the intrusive serialize methods are templates, but it certainly would solve the problem if it were possible.
*** I suspect that if the other changes suggested were implemented this would disappear as a problem. I don't think I've tried it, but rather than including boiler plate code in each derived class, one might try adding a "mix-in" base class which contains the serialize function.
In addition the bodies of all of my overrides are completely identical except for the classname (AssetModel, AssetTexture, etc).
Which means I'll be wrapping the bodies of a generic save_override and load_override in a macro, and have to manually add all Asset derived classes to a list of classes inside my archive class. Which means that my archiver cannot be generic, even though I have managed to make it a template in the sense that the passed in asset manager and base asset types are template parameters.
*** looks to me that you've gotten off on the wrong foot and stuck with it.
the type ResourceID is a std::string
saving... class asset_oarchive : ... void save_override(const AssetModel& t, int i) { if (!RootAssetSaved) { RootAssetSaved = true; binary_oarchive_impl::save_override(t, i); } else { AssetManagerType::ResourceID ResourceId = TheAssetManager->ResourceIdForAsset(&t); operator&(ResourceId); } }
loading ... class asset_iarchive : ... void load_override(AssetModel*& t, int i) { if (!RootAssetLoaded) { RootAssetLoaded = true; binary_iarchive_impl::load_override(t, i); } else { unsigned char c[10]; for (INT x=0; x<10; ++x) { c[x] = 0xFF; } for (INT x=0; x<4; ++x) { operator&(c[x]); //hackhack } AssetManagerType::ResourceID ResourceId; operator&(ResourceId); AssetType* a = TheAssetManager->FindAsset(ResourceId); if (!a) { a = TheAssetManager->LoadAsset(ResourceId); } // TODO: throw on missing resources, potentially query asset manager for substitutes t = static_cast
(a); } }
*** me this is exactly the wrong approach. Now you've coupled your classes to be serialized to a specific archive. This means you won't be able to use any other archive type and you've defeated one of the main benefits to the serialization library. Perhaps it wasn't a suitable library for your task. So I have a working implementation, how do I make it better? **** Maybe you might try doing it in the simplest way. I can't see how what you want to do is different than what everyone else uses the library for. And I can't see how what you want to do is different than what the examples do. Robert Ramey
_______________________________________________ Boost-users mailing list Boost-users@lists.boost.org http://lists.boost.org/mailman/listinfo.cgi/boost-users
I got the polymorphic behavior on serializing classes descending from my
root Asset class worked out, as well as the (tiny) changes required to get
the boost serialization library to load the objects my way :)
Basically it amounted to having to adding a set of symmetrical methods in
common_oarchive and common_archive, which the user-derived archive override.
common_oarchive:
template<class T>
bool save_object_ptr_custom(T& t)
{
return false;
}
common_iarchive:
template<class T>
bool load_object_ptr_custom(T& t)
{
return false;
}
pointer_oserializer::save_object_ptr -- wrap the last few lines with the
if implementing the callback:
if (!ar_impl.save_object_ptr_custom(t))
{
boost::serialization::save_construct_data_adl
participants (2)
-
Robert Ramey
-
Sean Cavanaugh