[rpc] Introducing Boost.Reflect, Boost.CMT, and Boost.RPC
I have just recently completed my first alpha release of a generic RPC library designed to support different protocols (JSON, XML, Protocol Buffers) and different transports (UDP, TCP, HTTP) in a generic manner. Currently only JSON over TCP is implemented. This library builds on top of my Boost.Reflect library which provides simple reflection along with a generic type-erasure construct, boost::reflect::any_ptr<T> Lastly all of this builds on top of my Boost.CMT (Collaborative Multi-Tasking Library) which seems to be along the same lines as Oliver's Boost.Strand library, but with a simpler interface. It is built on top of Boost.Context. This allows the RPC to be asynchronous, thread-safe, and free of nasty completion handlers. I have documentation of all three libraries posted here. http://bytemaster.github.com/dev/modules.html I would like to know if I am barking up the right tree here and if anyone has any feedback. If someone has a real project they would like to see this code incorporated into I would be more than willing to focus my development effort toward those needs.
2011/7/28 Daniel Larimer <dlarimer@gmail.com>:
I have just recently completed my first alpha release of a generic RPC library designed to support different protocols (JSON, XML, Protocol Buffers) and different transports (UDP, TCP, HTTP) in a generic manner.
Sounds great!
Lastly all of this builds on top of my Boost.CMT (Collaborative Multi-Tasking Library) which seems to be along the same lines as Oliver's Boost.Strand library, but with a simpler interface. It is built on top of Boost.Context. This allows the RPC to be asynchronous, thread-safe, and free of nasty completion handlers.
You don`t use asio, is you code portable to other operating systems? Can it be tuned, to use asio/libev?
I have documentation of all three libraries posted here.
I took a quick glance. Looks like "alpha release", but it is possible to understand the idea. This looks like not a safe approach: void operator()( Member Class::*p, const char* name ) You serialize no info about function parameters. I`m interested, but now I have no projects were it is possible to use your library. Best regards, Antony Polukhin
On Thu, Jul 28, 2011 at 2:49 PM, Antony Polukhin <antoshkka@gmail.com>wrote:
2011/7/28 Daniel Larimer <dlarimer@gmail.com>:
I have just recently completed my first alpha release of a generic RPC library designed to support different protocols (JSON, XML, Protocol Buffers) and different transports (UDP, TCP, HTTP) in a generic manner.
Sounds great!
Lastly all of this builds on top of my Boost.CMT (Collaborative Multi-Tasking Library) which seems to be along the same lines as Oliver's Boost.Strand library, but with a simpler interface. It is built on top of Boost.Context. This allows the RPC to be asynchronous, thread-safe, and free of nasty completion handlers.
You don`t use asio, is you code portable to other operating systems? Can it be tuned, to use asio/libev?
It uses Boost.ASIO for all networking. I just wrap Boost.ASIO async calls with Boost.CMT 'coroutine' handlers such that an async call becomes a future/promise instead of a callback. This way there is no need to worry about threading issues.
I have documentation of all three libraries posted here.
I took a quick glance. Looks like "alpha release", but it is possible to understand the idea.
This looks like not a safe approach: void operator()( Member Class::*p, const char* name ) You serialize no info about function parameters.
Depends upon which visitor you are using. If you are visiting all of the functors on a boost::reflect::vtable as used by boost::reflect::any_ptr<Interface> then Member is a boost::reflect::mirror_member<> specialization (a functor) and therefore Member::fused_params -> boost::fusion::vector<paramtypes> or Member::traits -> boost::function_traits. https://github.com/bytemaster/boost_reflect/blob/master/examples/cli.hppline 29 shows how to print out the function signature for the member. If you are using boost::reflector<T>::visit( visitor ) and you expected your visitor to handle member functors then your visitor would need specializations for each different number of parameters (or veridic templates). In practice I find that for interfaces I am using the vtable of functors in any_ptr<Interface> and for data types (structures) that I use as parameters I am only reflecting public members and not functors. For an example of how reflection serializes structs see: https://github.com/bytemaster/Boost.RPC/blob/master/include/boost/rpc/json.h... 94.
I`m interested, but now I have no projects were it is possible to use your library.
Best regards, Antony Polukhin _______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Hi, I am quite interested by boost::reflect : I have a few csv exports of existing structures to do, it sounds like the right tool (even though it can apparently do much more). So far it's the smallest effort to reflect existing data structures (no need to specify the types of members). I got "a" version (the "master" branch) from github and I tried the sample from http://bytemaster.github.com/dev/group__boost__reflect__quickstart.html but I had to change a few things from the code to make it work: - vtable.hpp line 47 vtable<T, InterfaceDelegate> ---> vtable<Visitor, InterfaceDelegate> - xml_printer::operator() must be const, it is not in the sample - you must use the "xml_printer" functor by explicitely specifying the template argument (xml_printer<my_struct>(t) instead of xml_printer(t)). I am using gcc 4.5.2 under Ubuntu in case that matters. Regards, Julien
On Jul 29, 2011, at 3:22 AM, Julien Nitard wrote:
Hi,
I am quite interested by boost::reflect : I have a few csv exports of existing structures to do, it sounds like the right tool (even though it can apparently do much more). So far it's the smallest effort to reflect existing data structures (no need to specify the types of members).
Great to hear, that is actually my primary goal with boost reflect, minimize extra typing.
I got "a" version (the "master" branch) from github and I tried the sample from http://bytemaster.github.com/dev/group__boost__reflect__quickstart.html but I had to change a few things from the code to make it work:
- vtable.hpp line 47 vtable<T, InterfaceDelegate> ---> vtable<Visitor, InterfaceDelegate>
vtable.hpp line 47 should never actually be instantiated unless you attempt to use it with a type that has not had the BOOST_REFLECT_ANY() macro specified for it. That said, the correct fix is vtable<InterfaceType,InterfaceDelegate>. I had changed the template parameter to be more descriptive for documentation purposes and missed that.
- xml_printer::operator() must be const, it is not in the sample
You are correct, documentation error. It has been updated.
- you must use the "xml_printer" functor by explicitely specifying the template argument (xml_printer<my_struct>(t) instead of xml_printer(t)).
Also correct. I have updated the example to reflect that and added a new helper to the example that should simplify things. xml_printer::make( t ) will deduce the type for you, useful if t is complex.
I am using gcc 4.5.2 under Ubuntu in case that matters.
Thanks for all of your input, changes have been checked into git and documentation updated. I have only tested on Mac OS X with gcc 4.5 and 4.2.1, limited testing on Windows. I suspect that Boost.Context will need a small CMake change to link the proper asm. On Windows you have to use the nmake build because a cmake bug prevents it from compiling the Boost.Context asm with the VS generators. More documentation is coming.
Regards,
Julien _______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
I would like to know if I am barking up the right tree here and if anyone has any feedback. If someone has a real project they would like to see this code incorporated into I would be more than willing to focus my development effort toward those needs.
The three libraries look really useful. The documentation is clear but a bit sparse. RPC lib: - Can you compare yours to other widely used libraries, e.g.Thrift CMT lib: - Can you compare your library to the use of coroutines in the Boostcon 2011 example https://github.com/chriskohlhoff/awesome - Could you provide a more real-life example, e.g. a proxy like in the above example ? - Can you provide an example that shows how to handle timeouts ? thanks jose
On Jul 29, 2011, at 7:39 PM, Jose wrote:
I would like to know if I am barking up the right tree here and if anyone has any feedback. If someone has a real project they would like to see this code incorporated into I would be more than willing to focus my development effort toward those needs.
The three libraries look really useful. The documentation is clear but a bit sparse.
There is so much to cover. I never got around to discussing how CMT does priority scheduling or too much detail into how real multi-threading works.
RPC lib: - Can you compare yours to other widely used libraries, e.g.Thrift
1) No code generation required (best for C++ to C++, but with a proper JSON interface can communicate with any language) - eliminate extra build steps - changing your code or structs in one spot applies every where. - expose any existing code as an RPC service in a non-intrusive manner. 2) Asynchronous operations performed via futures - Boost.CMT allows simple single-threaded programs to appear multi-threaded. - Benefits of async, with ease of use of synchronous code. - No need to have a real OS thread per 'async operation' nor block an OS thread waiting on a result. 3) Clean syntax, minimal macro usage 4) Automatic stubs, complete with full type erasure provided by boost::reflect::any_ptr<> - expose the same interface over JSON-RPC/TCP/UDP/HTTP, XML-RPC, protocol buffers, boost::serialization, etc - your algorithms interface with boost::reflect::any_ptr<> and so do not care if they are dealing with an RPC stub a local object, or a local actor running in a separate thread. 5) boost::signals over RPC 6) Asynchronous RPC call: Assuming you are using the mirror delegate then everything looks synchronous, but can be made asynchronous using boost::cmt boost::any_ptr<Interface> client_ptr; boost::cmt::async<ReturnType>( boost::bind(client_ptr->method_name, arg1, arg2 ) ).wait( timeout ); I have not yet implemented it, but boost::cmt::actor_interface will adapt all of the methods on an interface to return a future<> instead of the real return value. Thus: boost::any_ptr<Interface,boost::cmt::actor_interface> client_ptr; future<ReturnType> r = client_ptr->method_name( arg1, arg2 ); And of course, future<ReturnType> automatically casts to ReturnType and blocks (yields) if necessary. Then I would have a boost::actor_ptr<Interface> that could cast to and from a boost::any_ptr<Interface>. I am torn about making boost::cmt depend upon boost::reflect vs introducing yet another library called boost::actor and then having boost::rpc depend upon boost::actor.
CMT lib: - Can you compare your library to the use of coroutines in the Boostcon 2011 example https://github.com/chriskohlhoff/awesome
Awesome appears to provide stackless coroutines whereas CMT uses Boost.Context to provide a real stack per fiber.
- Could you provide a more real-life example, e.g. a proxy like in the above example ?
I use 'coroutines/fibers/futures' any time I need to perform an asynchronous operation and then it turns into a "synchronous" operation where all 'in-flight' tasks are multi-plexed on one real thread. When I actually want to do things in 'parallel' I use boost::cmt::async() and get the return value as a future. So I could enqueue 10 tasks, then wait for 10 results. I often spawn a long-running task that exits with a signal/flag and I 'join' it by waiting on its future. Other things I do is have one fiber 'wait' on a boost::signal for a value from another fiber. Lastly, any time I need to do inter-thread communication, I get the 'cmt::thread' and schedule an async task. The result is that all of my code is thread-safe and lock-free as long as the shared resource is only accessed from one thread with other threads delegating queries to it. (Much like boost::asio::strand keeps things 'thread safe'). The only thing I need to be careful about is long-running tasks that do not yield. They can block all other tasks in a particular thread. With boost CMT it is easy enough to spawn these tasks off into a real thread. I try to avoid any blocking system calls as they stall not only the current fiber, but all other tasks assigned to that thread.
- Can you provide an example that shows how to handle timeouts ?
cmt::future<ReturnType> f = boost::cmt::async<ReturnType>( Functor, priority ); try { ReturnType r = f.wait( timeout_us); } catch( const boost::cmt::future_wait_timeout& e ) { .. handle timeout here.... }
thanks jose _______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
On Sat, Jul 30, 2011 at 5:08 AM, Daniel Larimer <dlarimer@gmail.com> wrote:
I am torn about making boost::cmt depend upon boost::reflect vs introducing yet another library called boost::actor and then having boost::rpc depend upon boost::actor.
Maybe you could have actor, rpc, cmt be part of concurrency toolkit , much like asio does for async networking (reactor, proactor and async with threads). e.g. mpt for message passing toolkit I would assume with this library you could have the message passing programming benefits you get with Erlang.
Awesome appears to provide stackless coroutines whereas CMT uses Boost.Context to provide a real stack per fiber.
Maybe your library should also provide both approaches. There are some important shortcomings for stackful coroutines. Can you comment on those ?
- Could you provide a more real-life example, e.g. a proxy like in the above example ?
The only thing I need to be careful about is long-running tasks that do not yield. They can block all other tasks in a particular thread. With boost CMT it is easy enough to spawn these tasks off into a real thread. I try to avoid any blocking system calls as they stall not only the current fiber, but all other tasks assigned to that thread.
Having real life examples be part of the library would help users see the benefits much quicker. I am not clear what would be the most compelling examples (proxy or web server are programmed best with the proactor pattern or stackless coroutines) - How to spawn a normal thread for blocking operations like file IO - a Map-Reduce example on how to distribute tasks to different cores and different machines - a basic persistent http client example that shows how to do pipelining (not so easy to do with proactor) and saves results to files I hope you propose boost::reflect for review soon and then mature all the other bits in a library that enables other concurrency models and is a good brother for asio. Many thanks for sharing your work
On Jul 30, 2011, at 4:40 AM, Jose wrote:
On Sat, Jul 30, 2011 at 5:08 AM, Daniel Larimer <dlarimer@gmail.com> wrote:
I am torn about making boost::cmt depend upon boost::reflect vs introducing yet another library called boost::actor and then having boost::rpc depend upon boost::actor.
Maybe you could have actor, rpc, cmt be part of concurrency toolkit , much like asio does for async networking (reactor, proactor and async with threads). e.g. mpt for message passing toolkit Not a bad idea, but I like small, well-defined libraries as they are easier to review and contain dependencies. Perhaps the unified concept of 'message passing' where inter-thread, inter-process, and inter-machine are all handled 'in the same way' could work.
I would assume with this library you could have the message passing programming benefits you get with Erlang.
Yes. I have been using tools similar to these libraries (but not as clean or generic) and I can testify that attempting to develop any application that deals with asynchronous events or multiple threads is greatly enhanced and simplified by using CMT.
Awesome appears to provide stackless coroutines whereas CMT uses Boost.Context to provide a real stack per fiber.
Maybe your library should also provide both approaches. There are some important shortcomings for stackful coroutines. Can you comment on those ?
The main shortcoming is allocating stacks and actual context switching. Using stackless solutions requieres the user to manage their yield points. I have developed code using stackless approach and it helps, but does not solve the problem of ASIO callbacks happening in a separate thread and thus accessing shared state from an asio handler becomes problematic.
Many thanks for sharing your work _______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
On Jul 30, 2011, at 4:40 AM, Jose wrote:
I hope you propose boost::reflect for review soon and then mature all the other bits in a library that enables other concurrency models and is a good brother for asio.
I take it as a high complement that you consider boost::reflect worthy to be considered for review soon. I am willing to submit it if I can can get enough feedback on it. 1) Does the name describe what it does? 2) Does the library fully address the problem within the scope of practicality? 3) Other reflection attempts have not been accepted for one reason or another. - Boost.Reflect vs Boost.Mirror 4) Would a different name better 'reflect' what the library does? 5) Are there any features missing? 6) What must be done before it can be submitted? - license info, unit tests, etc? Would it be more aptly named for its type-erasure, vtable, any_ptr<>, interface than its simple visitor-based reflection? The library is ready for informal review now and my goal is to morph it into something that is ready for full review (as quickly as possible), so any feedback / direction would be very helpful.
Hi Daniel, On Sat, Jul 30, 2011 at 11:05 PM, Daniel Larimer <dlarimer@gmail.com> wrote:
On Jul 30, 2011, at 4:40 AM, Jose wrote:
I hope you propose boost::reflect for review soon and then mature all the other bits in a library that enables other concurrency models and is a good brother for asio.
I take it as a high complement that you consider boost::reflect worthy to be considered for review soon. I am willing to submit it if I can can get enough feedback on it.
Your RPC library certainly looks interesting, but I've a few questions/comments on the reflection part.
1) Does the name describe what it does?
Basically yes, but from a quick glance at the docs I'm missing a few things (see below). Generally from my own experience I can say that a reflection library should not start as the support tool for a particular task (RPC, serialization in this case). It should aim to be as general as possible, as there are other situations where reflection can be useful. Mirror (the C++03 and C++11 version) are the third and fourth reflection libraries I've written from scratch ;) The first was begun actually the same way as Boost.Reflect as a support for RPC and it worked well for this task but it had two main problems it was too polymorphic (= too slow) and too specific (= not usable for other things)
2) Does the library fully address the problem within the scope of practicality?
If its goal is only to be the support for RPC then yes. Have you considered using another more general reflection library (shameless plug) like Mirror ? :) In the past I preferred to work on it alone since I had my ideas about how to do certain things. But now it is in a state where a second (and preferably third, fourth, ...) opinion would be very helpful. If your are missing something in Mirror I'm sure we could get it done.
3) Other reflection attempts have not been accepted for one reason or another. - Boost.Reflect vs Boost.Mirror
Here should be probably said that Mirror (and AFAIK) Boost.Reflection were not submitted for review (yet) and this is the only reason for them not being accepted. I don't know about Jeremy but I'm planning to do that once it is ready. The compile-time reflection is basically finished (with a few things that need polishing and a few features that can be added later). The two biggest problems I see are that it heavily depends on C++11 and some quick start guide and step-by-step tutorials are missing from the docs.
4) Would a different name better 'reflect' what the library does? 5) Are there any features missing?
I couldn't find things like namespace member traversals, support for compile-time metaprogramming, reflection of specifiers (access type, linkage/storage class, etc.) examining/setting members with getter/setter functions (with side effects), just to name some, but maybe I just missed them in the docs.
6) What must be done before it can be submitted? - license info, unit tests, etc?
Would it be more aptly named for its type-erasure, vtable, any_ptr<>, interface than its simple visitor-based reflection?
The library is ready for informal review now and my goal is to morph it into something that is ready for full review (as quickly as possible), so any feedback / direction would be very helpful.
Best, Matus author of the Mirror reflection utilities http://kifri.fri.uniza.sk/~chochlik/mirror-lib/html/
On Sun, Jul 31, 2011 at 4:11 AM, Matus Chochlik <chochlik@gmail.com> wrote:
Hi Daniel,
On Sat, Jul 30, 2011 at 11:05 PM, Daniel Larimer <dlarimer@gmail.com> wrote:
On Jul 30, 2011, at 4:40 AM, Jose wrote:
I hope you propose boost::reflect for review soon and then mature all the other bits in a library that enables other concurrency models and is a good brother for asio.
I take it as a high complement that you consider boost::reflect worthy to be considered for review soon. I am willing to submit it if I can can get enough feedback on it.
Your RPC library certainly looks interesting, but I've a few questions/comments on the reflection part.
1) Does the name describe what it does?
Basically yes, but from a quick glance at the docs I'm missing a few things (see below). Generally from my own experience I can say that a reflection library should not start as the support tool for a particular task (RPC, serialization in this case). It should aim to be as general as possible, as there are other situations where reflection can be useful.
I can only agree with you in part. If the requirement to be as 'general as possible' makes the resulting solution less optimal for the most common use cases (RPC, Serialization, etc) then your API will be much more difficult for 99% of the use cases so that you can support the 1% special cases. This, I believe, is the primary difference in our design philosophy.
Mirror (the C++03 and C++11 version) are the third and fourth reflection libraries I've written from scratch ;) The first was begun actually the same way as Boost.Reflect as a support for RPC and it worked well for this task but it had two main problems it was too polymorphic (= too slow) and too specific (= not usable for other things)
2) Does the library fully address the problem within the scope of practicality?
If its goal is only to be the support for RPC then yes. Have you considered using another more general reflection library (shameless plug) like Mirror ? :)
Yes, but see prior comment regarding ease of use vs. general solution.
In the past I preferred to work on it alone since I had my ideas about how to do certain things. But now it is in a state where a second (and preferably third, fourth, ...) opinion would be very helpful. If your are missing something in Mirror I'm sure we could get it done
Perhaps if Boost.Mirror provided a "quick and dirty" macro that does what BOOST_REFLECT(type, (base)(base2),(member)(member2)...) does then I could leverage it. That said, Boost.Mirror currently depends upon C++0x features not available in Visual Studio. Also, even with a simplified macro, Boost.Mirror's greater complexity/flexibility mean significantly longer compile times. Ideally, the reflection should be light-weight enough that users automatically add it to everything so they can benefit from automatic serialization, json, debug output, and many other tools that can leverage reflection. If adding reflection doubled my compile times then that may be a show stopper for many people. When it boils down to it, most of my reflection for basic types amounts to a simple function defined by a macro: template<typename T, typename Visitor> visit( Visitor v ) { visit<BaseA>(v); visit<BaseB>(v); v( &T::name, "name" ); v( &T::member, "member" ); } Combined with specializing get_typeinfo to get a cross-platform alternative to typeid(T).name(); template<typename T> struct get_typeinfo { static const char* name() { return "ClassName"; } } And a few built-in helpers for vector,map,list,pair, etc.
.
3) Other reflection attempts have not been accepted for one reason or another. - Boost.Reflect vs Boost.Mirror
Here should be probably said that Mirror (and AFAIK) Boost.Reflection were not submitted for review (yet) and this is the only reason for them not being accepted. I don't know about Jeremy but I'm planning to do that once it is ready.
The compile-time reflection is basically finished (with a few things that need polishing and a few features that can be added later).
The two biggest problems I see are that it heavily depends on C++11 and some quick start guide and step-by-step tutorials are missing from the docs.
There is also a significant amount of 'reflection' provided by BOOST_FUSION_ADAPT_STRUCT macro. In fact, I suspect that with the addition of a 'name_of<tag>' that it would provide all of the runtime/compile time reflection necessary for doing serialization. Unfortunately, boost::fusion, and I suspect Boost.Mirror would introduce substantially more compile time overhead than Boost.Reflect when you consider the number of types / meta functions created/evaluated. The benefit of BOOST_FUSION_ADAPT_STRUCT is that your types are ready for use with Boost.Spirit. Also, BOOST_FUSION_ADAPT_STRUCT cannot be used to reflect methods.
4) Would a different name better 'reflect' what the library does? 5) Are there any features missing?
I couldn't find things like namespace member traversals, support for compile-time metaprogramming, reflection of specifiers (access type, linkage/storage class, etc.) examining/setting members with getter/setter functions (with side effects), just to name some, but maybe I just missed them in the docs.
Most of those things are not there because they were not in the design
spec. Based upon these comments and the fact that Boost.Mirror is far more robust than my code, I think the question becomes which is better: 1) Move Boost.Reflect into Boost.RPC 2) Provide a simpler macro interface to Boost.Mirror 3) Two different reflection libraries 4) Move BOOST_REFLECT_ANY() into a Boost.Erasure library, simplify Boost.Mirror macros. 5) Added a small bit of reflection to BOOST_FUSION_ADAPT_STRUCT The last thing I want as an end-user is to specify BOOST_REFLECT..., BOOST_MIRROR..., and BOOST_FUSION... for all of their types. I suspect there is a place for all 3 solutions as they all make different tradeoffs in performance, flexibility, and compile times.
6) What must be done before it can be submitted? - license info, unit tests, etc?
Would it be more aptly named for its type-erasure, vtable, any_ptr<>, interface than its simple visitor-based reflection?
The library is ready for informal review now and my goal is to morph it into something that is ready for full review (as quickly as possible), so any feedback / direction would be very helpful.
Best,
Matus
author of the Mirror reflection utilities http://kifri.fri.uniza.sk/~chochlik/mirror-lib/html/ _______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
On Mon, Aug 1, 2011 at 9:55 AM, Daniel Larimer <dlarimer@gmail.com> wrote:
The last thing I want as an end-user is to specify BOOST_REFLECT..., BOOST_MIRROR..., and BOOST_FUSION... for all of their types.
Indeed.
I suspect there is a place for all 3 solutions as they all make different tradeoffs in performance, flexibility, and compile times.
+1. My $0.02. --DD
Sorry for the late response. I've been busy with other things recently. On Mon, Aug 1, 2011 at 4:55 PM, Daniel Larimer <dlarimer@gmail.com> wrote:
On Sun, Jul 31, 2011 at 4:11 AM, Matus Chochlik <chochlik@gmail.com> wrote:
[...]
1) Does the name describe what it does?
Basically yes, but from a quick glance at the docs I'm missing a few things (see below). Generally from my own experience I can say that a reflection library should not start as the support tool for a particular task (RPC, serialization in this case). It should aim to be as general as possible, as there are other situations where reflection can be useful.
I can only agree with you in part. If the requirement to be as 'general as possible' makes the resulting solution less optimal for the most common use cases (RPC, Serialization, etc) then your API will be much more difficult for 99% of the use cases so that you can support the 1% special cases. This, I believe, is the primary difference in our design philosophy.
This is true if the "generality" of the solution really makes it visibly less efficient in the distinct use cases, which I believe is not the problem of Mirror. It is very possible that your solution compiles faster but in run time the differences *should* be negligible and also the difference in compilation times is probably not an order of magnitude or something like that I say "should", this of course needs to be measured :-). The advantage of a general solution is that you don't need to register your classes multiple times if you want to use them with multiple libraries which somehow use meta-data describing those classes. You register it once and those libraries can (transform and) reuse the meta-data provided by Mirror. I've tried this with the SOCI database library where you can automate the implementation of its TypeConversion<> template or with Boost.Serialization.
[...]
Perhaps if Boost.Mirror provided a "quick and dirty" macro that does what BOOST_REFLECT(type, (base)(base2),(member)(member2)...) does then I could leverage it. That said, Boost.Mirror currently depends upon C++0x features not available in Visual Studio. Also, even with a simplified macro, Boost.Mirror's greater complexity/flexibility mean significantly longer compile times. Ideally, the reflection should be light-weight enough that users automatically add it to everything so they can benefit from automatic serialization, json, debug output, and many other tools that can leverage reflection. If adding reflection doubled my compile times then that may be a show stopper for many people.
There are the quick registering macros which greatly simplify the registering see for example here: http://kifri.fri.uniza.sk/~chochlik/mirror-lib/html/doxygen/puddle/html/da/d... When the registering and reflection of functions is finished I'll add such quick-reg macros also for them. + there will be automatic registering. [...]
.
3) Other reflection attempts have not been accepted for one reason or another. - Boost.Reflect vs Boost.Mirror
Here should be probably said that Mirror (and AFAIK) Boost.Reflection were not submitted for review (yet) and this is the only reason for them not being accepted. I don't know about Jeremy but I'm planning to do that once it is ready.
The compile-time reflection is basically finished (with a few things that need polishing and a few features that can be added later).
The two biggest problems I see are that it heavily depends on C++11 and some quick start guide and step-by-step tutorials are missing from the docs.
There is also a significant amount of 'reflection' provided by BOOST_FUSION_ADAPT_STRUCT macro. In fact, I suspect that with the addition of a 'name_of<tag>' that it would provide all of the runtime/compile time reflection necessary for doing serialization. Unfortunately, boost::fusion, and I suspect Boost.Mirror would introduce substantially more compile time overhead than Boost.Reflect when you consider the number of types / meta functions created/evaluated. The benefit of BOOST_FUSION_ADAPT_STRUCT is that your types are ready for use with Boost.Spirit. Also, BOOST_FUSION_ADAPT_STRUCT cannot be used to reflect methods.
This is true, as already said above the advantage of a general solution is that you can reuse the reflected meta-data in situations which you might not have anticipated in the beginning. I didn't try this, but off the top of my head I don't see anything preventing you to do the same thing that the BOOST_FUSION_ADAPT_STRUCT does by transforming the meta-data provided by Mirror and use the them in code which uses Boost.Fusion without registering it again. [...]
I couldn't find things like namespace member traversals, support for compile-time metaprogramming, reflection of specifiers (access type, linkage/storage class, etc.) examining/setting members with getter/setter functions (with side effects), just to name some, but maybe I just missed them in the docs.
Most of those things are not there because they were not in the design spec.
Based upon these comments and the fact that Boost.Mirror is far more robust than my code, I think the question becomes which is better:
1) Move Boost.Reflect into Boost.RPC 2) Provide a simpler macro interface to Boost.Mirror 3) Two different reflection libraries 4) Move BOOST_REFLECT_ANY() into a Boost.Erasure library, simplify Boost.Mirror macros. 5) Added a small bit of reflection to BOOST_FUSION_ADAPT_STRUCT
I would agree. I admit that Mirror is just "not there yet". And in the future if you are interested we could add support for optionally getting the meta-data from Mirror if they're available.
The last thing I want as an end-user is to specify BOOST_REFLECT..., BOOST_MIRROR..., and BOOST_FUSION... for all of their types.
This is exactly what I'm trying to avoid :) There are many libraries which use some sort of "reflective" programming and their main problem usually is that they force you to register the same (very often overlapping) things multiple times if you want to use them all in one application.
I suspect there is a place for all 3 solutions as they all make different tradeoffs in performance, flexibility, and compile times.
[...] Regards, Matus
2011/7/28 Daniel Larimer <dlarimer@gmail.com>:
I have just recently completed my first alpha release of a generic RPC library designed to support different protocols (JSON, XML, Protocol Buffers) and different transports (UDP, TCP, HTTP) in a generic manner. Currently only JSON over TCP is implemented.
I would love Apache thrift integration in your rpc framework. -- Quiero ser el rayo de sol que cada día te despierta para hacerte respirar y vivir en me. "Favola -Moda".
On Aug 5, 2011, at 5:48 AM, Giorgio Zoppi wrote:
2011/7/28 Daniel Larimer <dlarimer@gmail.com>:
I have just recently completed my first alpha release of a generic RPC library designed to support different protocols (JSON, XML, Protocol Buffers) and different transports (UDP, TCP, HTTP) in a generic manner. Currently only JSON over TCP is implemented.
I would love Apache thrift integration in your rpc framework.
Are you looking for native thrift binary support? You should be able to use the thrift-generated base classes with boost::reflect::any_ptr and achieve proper abstraction. Looking over the thrift website I can see several advantages to my approach: 1) They use a large thread pool to achieve what I do with Cooperative multitasking 2) Generated code does not compile on windows?? (there appears to be a patch for it) I am unable to find their 'on-the-wire' binary protocol spec. My preferred approach would be to support thrift 'natively' and then to use reflection to 'generate' the thrift IDL file for other languages.
-- Quiero ser el rayo de sol que cada día te despierta para hacerte respirar y vivir en me. "Favola -Moda". _______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Hi Daniel, It seems that the latest update for boost::reflect can't be used with member functions anymore. If I define the following: struct Coin { void f() { } }; BOOST_REFLECT(Coin, (f)); GCC complains: "error: invalid use of non-static member function" (version 4.5, -std=c++0x) On a separate subject, I've been working a bit on a few additions, notably a way to "register" types (though I didn't go as far as planned) and some code to reflect enums. I'll be posting about it soon. Thanks for your feedback, Regards, Julien
participants (7)
-
Antony Polukhin -
Daniel Larimer -
Dominique Devienne -
Giorgio Zoppi -
Jose -
Julien Nitard -
Matus Chochlik