Extend boost binary serialization with varints encoding
Is it feasible to provide boost binary serialization with varint encoding capabilities? How does one achieve that? Can one just create template overload of "save" and "load" for unsigned integrals? anything else should be changed? [Read More]http://feeds.feedburner.com/~r/sizmek-blog/~6/1 [http://www.sizmek.com/Sizmek.png]http://www.sizmek.com/
On 11/22/2015 12:35 PM, Ernest Zaslavsky wrote:
Is it feasible to provide boost binary serialization with varint encoding capabilities?
Yes, see: http://www.boost.org/boost/multiprecision/cpp_int/serialize.hpp
On 11/22/15 7:13 AM, Bjorn Reese wrote:
On 11/22/2015 12:35 PM, Ernest Zaslavsky wrote:
Is it feasible to provide boost binary serialization with varint encoding capabilities?
Yes, see:
http://www.boost.org/boost/multiprecision/cpp_int/serialize.hpp
wow - I never saw that before - probably a good thing. It also seems a lot more complex that I would think it should be. seems to me using separate save/load functions would be more transparent to the user of the code. Also one could make things more obvious by specialization of the Archive argument for binary_?archive. I'm not sure about the trivial vs. non-trivial aspect. Still one variant would seem better than 3 to me. Maybe we're sometimes too clever. Robert Ramey
Looks like exactly what I need. Thanks! -----Original Message----- From: Boost-users [mailto:boost-users-bounces@lists.boost.org] On Behalf Of Bjorn Reese Sent: Sunday, November 22, 2015 5:14 PM To: boost-users@lists.boost.org Subject: Re: [Boost-users] Extend boost binary serialization with varints encoding On 11/22/2015 12:35 PM, Ernest Zaslavsky wrote:
Is it feasible to provide boost binary serialization with varint encoding capabilities?
Yes, see: http://www.boost.org/boost/multiprecision/cpp_int/serialize.hpp _______________________________________________ Boost-users mailing list Boost-users@lists.boost.org http://lists.boost.org/mailman/listinfo.cgi/boost-users
Ok, I came up with quite ugly solution (see on coliru http://coliru.stacked-crooked.com/a/ba39b5a242e5c69b). but it works, however it is ugly in the way that it is up to user to apply encode/decode on members. Moreover it is the only way to encode value. I was aiming to something broader, for example, in array or vector the size is stored in archive, it is size_t and it occupies 64 bit, in case of small data chunks it is complete waste of storage. Is there a way to force all primitive types of choice to use my serialization instead of standard one? I mean is there a way to encode this aforementioned size_t in array serialization without rewriting its implementation? -----Original Message----- From: Boost-users [mailto:boost-users-bounces@lists.boost.org] On Behalf Of Bjorn Reese Sent: Sunday, November 22, 2015 5:14 PM To: boost-users@lists.boost.org Subject: Re: [Boost-users] Extend boost binary serialization with varints encoding On 11/22/2015 12:35 PM, Ernest Zaslavsky wrote:
Is it feasible to provide boost binary serialization with varint encoding capabilities?
Yes, see: http://www.boost.org/boost/multiprecision/cpp_int/serialize.hpp _______________________________________________ Boost-users mailing list Boost-users@lists.boost.org http://lists.boost.org/mailman/listinfo.cgi/boost-users
On 11/23/15 11:21 AM, Ernest Zaslavsky wrote:
Ok, I came up with quite ugly solution (see on coliru http://coliru.stacked-crooked.com/a/ba39b5a242e5c69b). but it works, however it is ugly in the way that it is up to user to apply encode/decode on members. Moreover it is the only way to encode value. I was aiming to something broader, for example, in array or vector the size is stored in archive, it is size_t and it occupies 64 bit, in case of small data chunks it is complete waste of storage. Is there a way to force all primitive types of choice to use my serialization instead of standard one? I mean is there a way to encode this aforementioned size_t in array serialization without rewriting its implementation?
I think you could make a much simpler more elegant solution using serialization wrappers - look in the documentation. But the broader question requires more thought. It sounds to me that you'd like to change how the archive class itself behaves. The way to do this is to derived from the archive class which has mostly what you want and override the handling of the types you're interested in. If you make your override accept the base archive as a template parameter, you can make your override applicable to any existing archive. I believe that the documentation has examples on how to do that as well. Robert Ramey
Thanks Robert, will check both, however the second approach of deriving from base and adding handling it is not something simple that can be done as POC in a day or two. Will start with wrappers. Thanks again! -----Original Message----- From: Boost-users [mailto:boost-users-bounces@lists.boost.org] On Behalf Of Robert Ramey Sent: Monday, November 23, 2015 11:41 PM To: boost-users@lists.boost.org Subject: Re: [Boost-users] Extend boost binary serialization with varints encoding On 11/23/15 11:21 AM, Ernest Zaslavsky wrote:
Ok, I came up with quite ugly solution (see on coliru http://coliru.stacked-crooked.com/a/ba39b5a242e5c69b). but it works, however it is ugly in the way that it is up to user to apply encode/decode on members. Moreover it is the only way to encode value. I was aiming to something broader, for example, in array or vector the size is stored in archive, it is size_t and it occupies 64 bit, in case of small data chunks it is complete waste of storage. Is there a way to force all primitive types of choice to use my serialization instead of standard one? I mean is there a way to encode this aforementioned size_t in array serialization without rewriting its implementation?
I think you could make a much simpler more elegant solution using serialization wrappers - look in the documentation. But the broader question requires more thought. It sounds to me that you'd like to change how the archive class itself behaves. The way to do this is to derived from the archive class which has mostly what you want and override the handling of the types you're interested in. If you make your override accept the base archive as a template parameter, you can make your override applicable to any existing archive. I believe that the documentation has examples on how to do that as well. Robert Ramey _______________________________________________ Boost-users mailing list Boost-users@lists.boost.org http://lists.boost.org/mailman/listinfo.cgi/boost-users
On 11/23/2015 08:21 PM, Ernest Zaslavsky wrote:
Ok, I came up with quite ugly solution (see on coliru http://coliru.stacked-crooked.com/a/ba39b5a242e5c69b). but it works, however it is ugly in the way that it is up to user to apply encode/decode on members. Moreover it is the only way to encode value. I was aiming to something broader, for example, in array or vector the size is stored in archive, it is size_t and it occupies 64 bit, in case of small data chunks it is complete waste of storage. Is there a way to force all primitive types of choice to use my serialization instead of standard one? I mean is there a way to encode this aforementioned size_t in array serialization without rewriting its implementation?
I was assuming that by "varint" you wanted an integral type of arbitrary length, yet your example only uses the built-in integral types. If you only require the latter, and you do not care particularly about which binary format is used, then I suggest that you try the transenc serialization archives from: https://github.com/breese/trial.protocol This will select the shortest integral encoding based on the value. Documentation can be found below, although the transenc part has not been documented yet: http://breese.github.io/trial/protocol/index.html You can find plenty of examples in the test suites: https://github.com/breese/trial.protocol/blob/develop/test/transenc/oarchive... https://github.com/breese/trial.protocol/blob/develop/test/transenc/iarchive...
Thanks, will check it too -----Original Message----- From: Boost-users [mailto:boost-users-bounces@lists.boost.org] On Behalf Of Bjorn Reese Sent: Tuesday, November 24, 2015 11:49 AM To: boost-users@lists.boost.org Subject: Re: [Boost-users] Extend boost binary serialization with varints encoding On 11/23/2015 08:21 PM, Ernest Zaslavsky wrote:
Ok, I came up with quite ugly solution (see on coliru http://coliru.stacked-crooked.com/a/ba39b5a242e5c69b). but it works, however it is ugly in the way that it is up to user to apply encode/decode on members. Moreover it is the only way to encode value. I was aiming to something broader, for example, in array or vector the size is stored in archive, it is size_t and it occupies 64 bit, in case of small data chunks it is complete waste of storage. Is there a way to force all primitive types of choice to use my serialization instead of standard one? I mean is there a way to encode this aforementioned size_t in array serialization without rewriting its implementation?
I was assuming that by "varint" you wanted an integral type of arbitrary length, yet your example only uses the built-in integral types. If you only require the latter, and you do not care particularly about which binary format is used, then I suggest that you try the transenc serialization archives from: https://github.com/breese/trial.protocol This will select the shortest integral encoding based on the value. Documentation can be found below, although the transenc part has not been documented yet: http://breese.github.io/trial/protocol/index.html You can find plenty of examples in the test suites: https://github.com/breese/trial.protocol/blob/develop/test/transenc/oarchive... https://github.com/breese/trial.protocol/blob/develop/test/transenc/iarchive... _______________________________________________ Boost-users mailing list Boost-users@lists.boost.org http://lists.boost.org/mailman/listinfo.cgi/boost-users
participants (3)
-
Bjorn Reese -
Ernest Zaslavsky -
Robert Ramey