Re: [boost] [serialization] performance comparison
Robert Ramey writes:
brass goowy wrote:
B"H Robert Ramey writes:
Our experiments with 1.3 revealed that the single biggest time consumer with the binary archive was with the stream i/o. version 1.34 - is re-implemented in terms of std::streambuf rather than stream i/o. It is significantly faster because of this.
I hope to test with 1.34 and report what I find.
I'll be curious how much difference still exists.
I ran this cvs -z3 -d:pserver:anonymous@boost.cvs.sourceforge.net:/cvsroot/boost checkout boost and I think I got the development version of 1.35. I've done a little bit of testing with it. I've returned to Linux and compared saving a list<int> using release versions of the libraries (I forgot to mention that with the Windows tests.) The lists had 20000 ints. I'm using 4.0.3 of the gnu compiler and -O3 on all of these tests. 1.33.1 // I pass an ofstream to binary_oarchive takes ~5200 microseconds 1.35 // I pass a filebuf to the binary_oarchive here takes ~3900 microseconds Ebenezer Enterprises takes ~1200 microseconds The stripped 1.35 executable is ~22,600 bytes and the stripped EE version is ~8,200 bytes. [...]
Does the 1.34 binary_oarchive take a streambuf instead of an ofstream? Is this documented somewhere?
It can take a stream buf or or an ofstream. If passed a stream, the stream buf is used directly. This preserves the common archive interface while gaining benefits of avoiding stream i/o operators.
I double checked the CVS using the browser interface. 1.34 indeed includes updated documention and code which supports the streambuffer interface. It doesn't say much, because whether binary archives were implemented in terms of stream operators or streambuf calls was considered an implementation detail. The only difference from a user stand point is that one can create and use a streambuf without having to create a stream itself.
In our tests we found avoid the stream io and using streambuf calls directly decreased the time required for vectos and arrays of primitives by a factor of 4 (if I remember correctly) - when used with a large buffer.
I haven't tested with vectors yet. But with list<int> I didn't see a factor of 4. It wasn't quite a factor of 2 between 1.33.1 and 1.35. I know 1.35 could be in the middle of changes so I didn't want to do a bunch of tests on it. I'm not sure how to get a version of 1.34. I just ran that command above and built it and the libraries have 1_35 in the names... How do I get 1.34? [ ...]
These may not be the most common application, but the C++ middleware/serialization frameworks that I'm familiar with support container classes. From my perspective the tests I chose are basic things you have to have in order to support more complicated things.
I took a cursory look at the www.ebenezer.net and the documentation. If you want I can include a pointer to it in the introduction to the serialization library - as I have with other libraries which have been brought to my attention.
That is up to you. But the address above is incorrect. The correct address is below. Shalom Brian Wood www.webEbenezer.net _______________________________________
brass goowy wrote:
Robert Ramey writes:
1.33.1 // I pass an ofstream to binary_oarchive takes ~5200 microseconds
1.35 // I pass a filebuf to the binary_oarchive here takes ~3900 microseconds
Hmm I'm very surprised that there is any detectable difference between the above two.
Ebenezer Enterprises takes ~1200 microseconds
We'll look into this. I'm surprised to find such a difference. The serialization code for standard containers of primitives is just a loop through the container elements which writes to the streambuf. (unless I forgot something or there is bug - always a possibility) Its not clear to me how to make this much faster without setting aside std::streambuf and using some other less portable method. In my view, boost bjam doesn't provide easy access to code profiling which is what I think is needed here. Robert Ramey
The stripped 1.35 executable is ~22,600 bytes and the stripped EE version is ~8,200 bytes.
This is less of a concern to me. boost serialization uses a lot of template code which may or may not account for this. I don't know how comparable the feature sets are and its possible that this might explain such a difference.
I haven't tested with vectors yet.
Your tests with std::list have already convinced me that it's worth looking into. Robert Ramey
Robert Ramey wrote:
In my view, boost bjam doesn't provide easy access to code profiling which is what I think is needed here.
What would easy access to profiling be? How is building with "variant=profile" different from easy access? -- -- Grafik - Don't Assume Anything -- Redshift Software, Inc. - http://redshift-software.com -- rrivera/acm.org - grafik/redshift-software.com -- 102708583/icq - grafikrobot/aim - grafikrobot/yahoo
Is this documented anywhere? My current preferred development platform is VC 7.1 . In 7.0 Microsoft dropped support for an execution profile in favor of another system which is much less convenient to use. Does BJAM support this? So I guess I should have said "to my knowledge" rather than "in my view". Sorry about that. Robert Ramey Rene Rivera wrote:
Robert Ramey wrote:
In my view, boost bjam doesn't provide easy access to code profiling which is what I think is needed here.
What would easy access to profiling be? How is building with "variant=profile" different from easy access?
Robert Ramey wrote:
Is this documented anywhere?
Hm, I don't know where the BBv2 syntax I mentioned is documented. But the tutorial does talk about an even simpler syntax of just adding "debug" or "release" <http://boost.org/boost-build2/doc/html/bbv2/tutorial.html>. And the front page mentions the profile variant <http://boost.org/boost-build2/>. Aha, here's the exact docs <http://boost.org/boost-build2/doc/html/bbv2/advanced/builtins/features.html>.
My current preferred development platform is VC 7.1 . In 7.0 Microsoft dropped support for an execution profile in favor of another system which is much less convenient to use. Does BJAM support this?
I don't know. First of, I've never used the VC profile support so I could not even begin to guess what you mean. But looking at the toolset sources, for msvc it doesn't seem to add any options to the commands when profiling=on.
So I guess I should have said "to my knowledge" rather than "in my view". Sorry about that.
No need to apologize :-) I'm about as uninformed as you in this respect. Hence why I'm asking what's wanted/needed/available. -- -- Grafik - Don't Assume Anything -- Redshift Software, Inc. - http://redshift-software.com -- rrivera/acm.org - grafik/redshift-software.com -- 102708583/icq - grafikrobot/aim - grafikrobot/yahoo
participants (3)
-
brass goowy -
Rene Rivera -
Robert Ramey