Re: [Boost-users] Fastest way of serializing a huge vector of ints into a human readable string

Jens Weller wrote:

Hm, maybe boost::spirit::karma could do you a favor here, but I think it will not gain much perfomance, if at all.

Maybe you could think about a functor which does it on your own. Do the numbers repeat often, then you could also build a table to only convert once to string.

Also parellizing with boost Thread would be an option.

Hi Jens, hmm. Documentation about boost::spirit::karma seems to be well hidden. Even a google search like karma site:www.boost.org/doc/libs/1_38_0/libs/spirit did not reveal any hits. Can you give me a link? Writing an own functor might be an option, I guess. The numbers do not repeat often, though. Quite the opposite: It is for certain that every number appears just once. And it could be anything within a 64bit range. Parallelization certainly could be helpful in some scenarios, but I a have a multi-threaded application already. So splitting up the serialization might increase speed for the serialization itself, while slowing down the total process due to increased context switches. I therefore want to have the single threaded operation as efficient as possible. Regards, Roland

mfg.

Jens Weller

-------- Original-Nachricht --------

...

Datum: Fri, 27 Feb 2009 18:46:07 +0100 Von: Roland Bock An: boost-users@lists.boost.org Betreff: [Boost-users] Fastest way of serializing a huge vector of ints into a human readable string

...

Hi,

I have a program which produces a vector of integers (several million entries). I need to write that into a human-readable string of space separated numbers.

I wonder, what would be the fastest way?

My first attempt was

stringstream resultStream; copy(integers.begin(), integers.end(), ostream_iterator<int>(resultStream, " ")); string result = resultStream.str();

But that requires copying the string.

Thus I was wondering if I could use boost to write to the target string directly?

boost::iostreams allow me to do the following:

string result; boost::iostreams::filtering_ostream out(boost::iostreams::back_inserter(result)); copy(integers.begin(), integers.end(), ostream_iterator<uint64>(out, " "));

This is faster by almost a factor of 2.

Any ideas how to increase speed even more?

Thanks in advance,

Roland _______________________________________________ Boost-users mailing list Boost-users@lists.boost.org http://lists.boost.org/mailman/listinfo.cgi/boost-users