Ruediger Berlich writes:
> Hi there,
>
> I am in the process of speeding up communication between a server and its
> clients. Communication involves serialized class data. Messages can be as
> large as 100 kilobytes.
> I have done some measurements which have shown that, in a cluster with
> Gigabit networking, most overhead of the parallelisation seems to come from
> the Broker infrastructure and the process of (de-)serialization. Network
> latency and/or bandwidth seems to play only a minor role in this
> environment.
> Hence, apart from optimizing my broker, I'm looking for ways to optimize the
> serialization process, as used in my application. As messages are discarded
> as soon as they reach the recipient, versions of serialized data do not play
> an important role.
I don't find any mention of "message" in the 1.42 Boost
Serialization documentation. Are you using MPI?
The obvious thing you've not mentioned is compression. I use
bzip2 to compress and decompress data -
http://webEbenezer.net/misc/SendCompressedBuffer.hh
http://webEbenezer.net/misc/ReceiveCompressedBuffer.hh
Besides using bzip2, I use variable-length integers to encode
the size of the compressed data. It works to use a fixed-size
integer as well, but frequently you can shave a couple of
bytes off the total by using a variable-length integer.
Brian Wood
http://webEbenezer.net
(651) 251-9384
"The kingdom of heaven is like a treasure hidden
in the field, which a man found and hid again;
and from joy over it he goes and sells all that
he has and buys that field."