Howabout compressing the whole stream in a general way.
Why not encrypt it while you're at it.
Oh and avoid wrtting code altogether.
I bet you think I'm being sarcastic.
Well, I'm not - I'm serious.
I would investigate using the serialization library along with binary (or maybe text as an experiment), in combination of io streams library. Serialization should work with ANY stream. And the io streams library includes zlib compression. (I don't know about encryption). So in theory, you should be able to get your data stream shrunk by 50% and free encryption - without writting any code at all.
Absolutely, compression will be a consideration, but does it make sense not to optimize the container before compression? Having lots of null bytes should compress very well, but having no null bytes will compress even better... :) There's also the matter of compression and decompression time to take into account. The tiny handheld computers that this will be deployed on are very constrained in terms of CPU and battery life. It's possible that the additional compression and decompression will shave some battery life off the device, and we need all we can get. Additionally, getting zlib and bzip to compile on Windows didn't work "out of the box" for me, and I didn't take the time to fight it yet, so compression simply wasn't available for my initial testing.