Re: [boost] Is there any interest in a library for actor programming? serialization + uniform names
The serialization and uniform name really boils down to the uniform_type_info class: Uniform names are simply a necessity if you need to identify a type based on its name. The goal is to send any kind of value to a remote machine and being able to deserialize it. If you have platofrm-independent names, you can simply prefix the binary serialized value with its type name and then get the RTTI (uniform_type_info) from the type name and deserialize the rest of the data. Boost.Actor is a bit more cleverer than that and maintains a list of type ids for each node it's connected to. Whenever you send a message with a type you didn't use beforehand, you send the type name along with its newly generated, unique ID and only use that ID going forward. But that's just an optimization. When talking about stateless communication (e.g. UDP), you can't do this and have to generate self-consistent messages. The fact that we even have to do this, is because std::type_info::name() is utterly useless. It returns a mangled name on GCC/Clang and a more sophisticated version of the type name on MSVC. The uniform name is the type name as it appears in code. So the first part of generating the uniform name is to demangle the platform-dependent name (https://github.com/Neverlord/boost.actor/blob/master/src/demangle.cpp). After that, we need to decompose the whole thing and make sure that qualifiers are always in the same order etc (https://github.com/Neverlord/boost.actor/blob/master/src/to_uniform_name.cpp). All this really is only aiming at having at type_info that is actually useful, i.e., name() returns *the exact same string* on each platform and it provides you with an interface for (1) generating values, (2) serializing and deserializing values, and (3) compare values. When generating a value through an uniform_type_info, you'll get a uniform_value, which is bascially a wrapper keeping {uniform_type_info*, void*}. Since we need a way to pass a serializer to a uniform_type_info instance, we can't use a templated approach as the archives in boost.serialization. The serialization classes in use (https://github.com/Neverlord/boost.actor/blob/master/boost/actor/serializer....) operate on sequences and primitive types. With the begin_object member function you also have a way of define serialization recursively, but I don't know if it's really needed to be honest. The write_tuple member function is probably not needed either, since you can simply just write the values one by one. However, it does give you a way of grouping values representing a single member together, which can be quite helpful when serializing to strings for instance. By the way, each announced type in Boost.Actor implicitly has a to_string(), which simply passes the value to a string serializer. There's also a from_string() that passes the input to a string deserializer, which gives you a simple way of injecting messages to the system. But the whole string serialization is not really fleshed out and the format not well defined. ________________________________________ From: Boost [boost-bounces@lists.boost.org] on behalf of Hartmut Kaiser [hartmut.kaiser@gmail.com] Sent: Tuesday, May 13, 2014 20:46 To: boost@lists.boost.org Subject: Re: [boost] Is there any interest in a library for actor programming? serialization + uniform names I'd be interested in hearing more details about this. Regards Hartmut --------------- http://boost-spirit.com http://stellar.cct.lsu.edu
-----Original Message----- From: Boost [mailto:boost-bounces@lists.boost.org] On Behalf Of Andrzej Horoszczak Sent: Tuesday, May 13, 2014 8:16 AM To: boost@lists.boost.org Subject: Re: [boost] Is there any interest in a library for actor programming? serialization + uniform names
_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
On Wed, May 14, 2014 at 9:24 AM, Charousset, Dominik
The serialization and uniform name really boils down to the uniform_type_info class:
Uniform names are simply a necessity if you need to identify a type based on its name. The goal is to send any kind of value to a remote machine and being able to deserialize it. If you have platofrm-independent names, you can simply prefix the binary serialized value with its type name and then get the RTTI (uniform_type_info) from the type name and deserialize the rest of the data.
This is precisely why I requested such a uniform type name feature during the typeindex review, but many didn't seem interested in that, and didn't want typeindex to provide it, mainly punting to a future "library" on top of typeindex. I think it's unfortunate, but since you seem to have solved this, I hope this could be revisited in light of this new info. Thanks, --DD
I think having something like uniform_type_info in Boost would be really nice. But it also conflicts with boost.serialization in a way. The "uniform type conversion" itself is quite messy (as you'll see for yourself in the sources). And it's expensive due to all the string operations, so you really want to make sure this is performed only once and then stored in a singleton of some description.
________________________________________
From: Boost [boost-bounces@lists.boost.org] on behalf of Dominique Devienne [ddevienne@gmail.com]
Sent: Wednesday, May 14, 2014 09:35
To: boost@lists.boost.org
Subject: Re: [boost] Is there any interest in a library for actor programming? serialization + uniform names
On Wed, May 14, 2014 at 9:24 AM, Charousset, Dominik
The serialization and uniform name really boils down to the uniform_type_info class:
Uniform names are simply a necessity if you need to identify a type based on its name. The goal is to send any kind of value to a remote machine and being able to deserialize it. If you have platofrm-independent names, you can simply prefix the binary serialized value with its type name and then get the RTTI (uniform_type_info) from the type name and deserialize the rest of the data.
This is precisely why I requested such a uniform type name feature during the typeindex review, but many didn't seem interested in that, and didn't want typeindex to provide it, mainly punting to a future "library" on top of typeindex. I think it's unfortunate, but since you seem to have solved this, I hope this could be revisited in light of this new info. Thanks, --DD _______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
On 14 May 2014 at 7:24, Charousset, Dominik wrote:
All this really is only aiming at having at type_info that is actually useful, i.e., name() returns *the exact same string* on each platform and it provides you with an interface for (1) generating values, (2) serializing and deserializing values, and (3) compare values. When generating a value through an uniform_type_info, you'll get a uniform_value, which is bascially a wrapper keeping {uniform_type_info*, void*}.
Implementing this correctly would be quite a bit of work which is why it was left out of TypeIndex, but very valuable I think. You need to write a mangled symbol to partial AST parser for MSVC and GCC and then you can compare the ASTs for equivalence. Here is one based on Boost.Spirit which parses MSVC mangling: https://github.com/ned14/NiallsCPP11Utilities/blob/master/SymbolMangle rMSVC.cpp. It is fundamentally broken however because unlike the Itanium mangling, the MSVC mangling is context sensitive i.e. its grammar is non-trivial, so a working design would need at least a two pass parser, not the single pass design here. Someone may suggest you simply lift the mangled symbol to pretty name code from clang as there should be both MSVC and Itanium ABI implementations in there, and the pretty names might have similar whitespacing etc. That might work, though be aware that I did not find a fully correct and working MSVC demangler (usually called __unDName) in any open source code when I last looked two years ago. The MSVC demangling process needs a full two pass parser, anything less won't cut it for really complex nested template template types. The implementation for mingw or ReactOS simply does (did) not work. Niall -- ned Productions Limited Consulting http://www.nedproductions.biz/ http://ie.linkedin.com/in/nialldouglas/
On 15/05/2014 01:11, Niall Douglas wrote:
(...) The MSVC demangling process needs a full two pass parser, anything less won't cut it for really complex nested template template types. (...)
Hi Niall, Would you have an example of a case where this is needed, off the top of your head? Thanks, MAT.
On 18 May 2014 at 7:59, Mathieu Champlon wrote:
The MSVC demangling process needs a full two pass parser, anything less won't cut it for really complex nested template template types.
Would you have an example of a case where this is needed, off the top of your head?
All the examples I knew of were written whilst I was at BlackBerry, and therefore are under NDA. Try trawling the bug reports at winehq for demangle failures: http://bugs.winehq.org/buglist.cgi?bug_status=__all__&content=demangle &no_redirect=1&order=relevance%20desc&product=&query_format=specific http://bugs.winehq.org/buglist.cgi?bug_status=__all__&content=undname& no_redirect=1&order=relevance%20desc&product=&query_format=specific The hardest part with the MSVC mangling scheme is dealing correctly with nested template templates combined with their symbol compression scheme (the ten most repeated symbols get compressed into their compressed index notation), because the template templates must be marked out during a first pass and then recursed into (you can't do a single forward pass with backtracking like with Itanium). Another gotcha is that arrays and pointers sometimes have their meaning inverted depending on context due to a historic bug in their mangler. Niall -- ned Productions Limited Consulting http://www.nedproductions.biz/ http://ie.linkedin.com/in/nialldouglas/
participants (4)
-
Charousset, Dominik -
Dominique Devienne -
Mathieu Champlon -
Niall Douglas