New library for RPC-calls using only boost and C++0x
Hello boost users and developers,
i'm currently writing a library for making remote procedure calls to
C++ objects. The library is based on the variadic templates extension
coming with C++0x (already present in GCC 4.3) in conjunction with the
boost library (threads, serialization, asio). The aim is to have a
library, that is easy to integrate and doesn't require additional
tools. Writing it, so far, has thaught me a lot about C++, templates,
variadic templates, threading, networking etc..
I post this message, because i think that this library is quite handy
in situations, where a developer already uses boost libraries and
wants to add some networking functionality to his/her project and the
usage of variadic templates instead of additional tools is quite new.
One example would be remote monitoring an application by adding a
"std::string get_status()"-function to some object. Implemented as
example is a chat-system with a server and clients interacting by
remote calls. This example is tested with a x64-linux and win32 using
the portable boost-text-serialization. Multiple clients can connect to
the server via the TCP/IP-acceptor (and disconnect without crashing
the system). Implemented is a console-client for text input and
display and a morse client, that interacts via a digital output device
(relais card), which is connected via a serial port. The serial port
and the digital output driver are both also connected via the
rpc-system and can be physically located on separate computers.
At this point, i have the basic bits together and a working
rpc-library, that needs to be cleaned, refined and extended to match
the practical needs. This means, that some programs and libraries have
to be written, which use the rpc-library and show its bugs and
limitations. The downside of it is, that the interface of the
rpc-library may change one more time until it can be considered
stable. Therefore it would not be good to test it widely in libraries
posing similar requirements, but to test it with a variety of possible
applications and provide the implementations as examples and
(regression-/unit-)tests.
- One library for Measurement/Automation is already in development.
- The chat-server/client exists.
- A transaction based game server is under consideration.
- Some distributed parallel computing application with dynamic
joining/leaving of clients.
Can you give me some more ideas of different applications, which are
easy to implement but yet useful?
What would you like to have implemented and accessible by an rpc-library?
One speed measurement using local (shared memory) channels resulted in
20000 calls/s and 60000 messages/s on a 1.8GHz Opteron. Do you know
how this compares to other implementations on similar hardware?
The TCP/IP-channel implementation using ASIO resulted in only about
100 synchronous calls/second regardless of the payload and almost no
CPU load, which suggests that ASIO somewhere "sleeps" on linux systems
for a few milliseconds. This would be fixed in some future
implementation. Using asynchronous calls (1000 open calls) again leads
to about 10000 calls/s.
I'll have a closer look on what can be merged with "channels" from the
boost vault[1], but from the first look it seems that "channels"
targets a much more general approach. After the first tries, i
realized, that my implemention has to be very specific (e.g. templates
only where necessary). So, the design approaches might be too
different.
After the refinement, cleanup, some testing and documenting, i'd like
to publish the rpc-library (and libraries based on it) under GPL- or
boost-license. Then you'll hear from me again. In future i'll probably
work 2-8 hours per week on the library and use it in some projects,
but there's plenty of work left to do in various regions. This will
leave space for other developers who might want to use and modify it
to fit their needs. Some of the harder parts that would need
implementation are redundant parallel connections and advanced
routing, authentication of nodes, signing of messages, encryption of
payload, access restriction to objects.
Please let me know if
* you are interested in this library in some way
* have comments
* know very similar libraries or
* are interested in serialization using variadic function templates.
Regards,
Siegfried Kettlitz
---
A short introduction on the library:
* "nodes" are the central parts. They contain a factory for objects,
store objects and forward "messages" to objects or other nodes in the
network and contain a "scheduler" for executing function calls. The
nodes are connected by "channels", which forward messages between the
two connected nodes. Currently there is a TCP/IPv4 implementation
using ASIO and direct forwarding.
* Nodes connected by channels form a network, create and monitor
virtual connections between each other. The network map is used to
route messages between connected nodes.
* An object has to supply a template for a class name (only if it can
be constructed by the factory) and a template for registering the
callable functions (also for abstract base classes which can't be
constructed).
* A function call is executed asynchronously (wrapper for synchronous
call exists). The call to node.call(...) returns a
"link_caller"-object.
The function to be executed itself can optionally have "link_callee"
as the first argument (which is invisible for the caller). The
"link_caller"/"link_callee" is used for synchronization (wait for call
to begin/finish execution), message forwarding (push/pop strings at
caller/callee) and transfer/extraction of the return value.
* Objects of class hierarchies with abstract base classes are possible
and handled transparently.
* Objects can be constructed by the factory of a node or an existing
(externally constructed) object can be registered by the node.
* Objects don't necessarily need to know (be modified, inherit
something etc.) that they are part of the RPC-system and some of their
functions are called remotely.
* Objects can be aware that they are constructed by the factory and
can, on construction be informed about their identity and get a
reference to the node to make remote calls themselves.
* An object is identified by a "reference" consisting of the "node_id"
and the "object_id".
* Parameters are passed (serialized, deserialized) by value. All
function parameters must have the possibility to be serialized.
Modifying referenced parameters won't affect the parameter at the
caller. Pointers obviously won't work between nodes not sharing the
same memory-address-space.
* Called functions can use "link_callee" to make remote calls
themselves, regardless of the object.
* Priorities are assigned to calls and affect message forwarding and scheduling.
* Calls to objects can be transformed into a boost::function using
node.bind(...).
* If a called function throws, the exception is forwarded as
rpc::exception or a more specialized exception class. The exception is
thrown at the caller, when waiting for interaction with the callee
(e.g. wait for return value).
* Template functions as targets are possible when specifying the
template parameters of the function pointer when registering the
function.
* There is no need to use macros at all, although macros could be used
to simplify some things.
----------
// Example for creating and calling an object.
// Create a service.
boost::asio::io_service node_io_service;
// Create a node.
rpc::node node_s(node_io_service, config);
// Register the builder in order to build the server object later.
node_s.register_builderdevicelib::chat_server_impl();
// Register the interface of the client to be able to make calls to it.
node_s.register_interfacedevicelib::chat_client();
// Create the server object.
rpc::reference cs( node_s.get_id(),
node_s.allocate_object_templatedevicelib::chat_server_impl() );
// Store the server in name server.
// rpc::reference( node_s.get_id(),
rpc::oid_simple_object_name_server ) -> A reference to the
object_name_server.
// rpc::sched_default() -> Default scheduling parameters.
// &rpc::simple_object_name_server::store, cs -> function to call.
// cs, "chat_server_object" -> function parameters.
node_s.wrap_call( rpc::reference( node_s.get_id(),
rpc::oid_simple_object_name_server ), rpc::sched_default(),
&rpc::simple_object_name_server::store, cs, "chat_server_object" );
----------
// Example for abstract base class.
namespace devicelib {
class chat_server : public chat_types
{
public:
virtual void receive( std::string xml_message ) = 0;
virtual user_id_t connect_user( rpc::reference a_client,
std::string xml_user_info ) = 0;
virtual std::string query_user( user_id_t id ) = 0;
virtual void disconnect_user( user_id_t user ) = 0;
virtual void join_room( user_id_t user, room_id_t room ) = 0;
virtual std::list
Hello Siegfried, On Thu, Jun 11, 2009 at 8:27 PM, Siegfried Kettlitz < siegfried.kettlitz@googlemail.com> wrote:
Can you give me some more ideas of different applications, which are easy to implement but yet useful? What would you like to have implemented and accessible by an rpc-library?
your lib can be of a great value. I have some may be too simple questions :) Which data representation format have you choosen? Is it XDR or smth. else proprietare. How will your lib bahave in mixed 32bit/64bit environments? Regards, Ovanes P.S. Do you know ICE (http://www.zeroc.com)? Until now I find it the most interesting midleware environment. It also supports mixed environmens: C++, .NET, Java, Python.
your lib can be of a great value. I have some may be too simple questions :) Which data representation format have you choosen? Is it XDR or smth. else proprietare.
It's proprietary so far. Each message has a header encoded in binary format (portable datatypes like uint16_t with adjusted endianness) and a body containing the serialized data. The current approach seems to be portable and fast. Maybe it makes sense to change the encoding, when a stable version is reached and compatibility to older versions becomes important.
How will your lib bahave in mixed 32bit/64bit environments?
Using the text archive of boost.serialization worked for the chat client under win32 to communicate with a linux64 server, while the non-portable binary archive type didn't work since it's not portable. It is important for me that different architectures can communicate because that's one important reason for having remote calls after all. It will work if the types match (or can be handles properly by boost.serialization). So "uint32_t" or "uint64_t" as function parameter will work while "unsigned int" won't work if not handled by serialization. Using a text archive should translate between those types if the values fit in the range of the smaller type. Little/Big-Endian issuses should be handled also, but remain untested, so far, as big-endian machines seem to get rare.
your lib can be of a great value. I have some may be too simple questions :) Which data representation format have you choosen? Is it XDR or smth. else proprietare.
It's proprietary so far. Each message has a header encoded in binary format (portable datatypes like uint16_t with adjusted endianness) and a body containing the serialized data. The current approach seems to be portable and fast. Maybe it makes sense to change the encoding, when a stable version is reached and compatibility to older versions becomes important.
To me the transport layer is more like a strategy pattern. So, using boost::serialization archives suits this well, but also you should think of the option to implement other protocols (XMLRPC f.e.). So that the user can extend this to its needs, or use one of the prebuild strategies. regards Jens Weller -- GMX FreeDSL Komplettanschluss mit DSL 6.000 Flatrate und Telefonanschluss für nur 17,95 Euro/mtl.!* http://portal.gmx.net/de/go/dsl02
There might be one minor problem with this. Visual Studio 2010 isn't going to support Variadic Templates. http://connect.microsoft.com/VisualStudio/feedback/ViewFeedback.aspx?Feedbac... If it arrives in version 11, that means Visual Studio 2012. I think the library sounds fantastic though. Damien Siegfried Kettlitz wrote:
Hello boost users and developers,
i'm currently writing a library for making remote procedure calls to C++ objects. The library is based on the variadic templates extension coming with C++0x (already present in GCC 4.3) in conjunction with the boost library (threads, serialization, asio). The aim is to have a library, that is easy to integrate and doesn't require additional tools. Writing it, so far, has thaught me a lot about C++, templates, variadic templates, threading, networking etc..
I post this message, because i think that this library is quite handy in situations, where a developer already uses boost libraries and wants to add some networking functionality to his/her project and the usage of variadic templates instead of additional tools is quite new. One example would be remote monitoring an application by adding a "std::string get_status()"-function to some object. Implemented as example is a chat-system with a server and clients interacting by remote calls. This example is tested with a x64-linux and win32 using the portable boost-text-serialization. Multiple clients can connect to the server via the TCP/IP-acceptor (and disconnect without crashing the system). Implemented is a console-client for text input and display and a morse client, that interacts via a digital output device (relais card), which is connected via a serial port. The serial port and the digital output driver are both also connected via the rpc-system and can be physically located on separate computers.
At this point, i have the basic bits together and a working rpc-library, that needs to be cleaned, refined and extended to match the practical needs. This means, that some programs and libraries have to be written, which use the rpc-library and show its bugs and limitations. The downside of it is, that the interface of the rpc-library may change one more time until it can be considered stable. Therefore it would not be good to test it widely in libraries posing similar requirements, but to test it with a variety of possible applications and provide the implementations as examples and (regression-/unit-)tests.
- One library for Measurement/Automation is already in development. - The chat-server/client exists. - A transaction based game server is under consideration. - Some distributed parallel computing application with dynamic joining/leaving of clients.
Can you give me some more ideas of different applications, which are easy to implement but yet useful? What would you like to have implemented and accessible by an rpc-library?
One speed measurement using local (shared memory) channels resulted in 20000 calls/s and 60000 messages/s on a 1.8GHz Opteron. Do you know how this compares to other implementations on similar hardware?
The TCP/IP-channel implementation using ASIO resulted in only about 100 synchronous calls/second regardless of the payload and almost no CPU load, which suggests that ASIO somewhere "sleeps" on linux systems for a few milliseconds. This would be fixed in some future implementation. Using asynchronous calls (1000 open calls) again leads to about 10000 calls/s.
I'll have a closer look on what can be merged with "channels" from the boost vault[1], but from the first look it seems that "channels" targets a much more general approach. After the first tries, i realized, that my implemention has to be very specific (e.g. templates only where necessary). So, the design approaches might be too different.
After the refinement, cleanup, some testing and documenting, i'd like to publish the rpc-library (and libraries based on it) under GPL- or boost-license. Then you'll hear from me again. In future i'll probably work 2-8 hours per week on the library and use it in some projects, but there's plenty of work left to do in various regions. This will leave space for other developers who might want to use and modify it to fit their needs. Some of the harder parts that would need implementation are redundant parallel connections and advanced routing, authentication of nodes, signing of messages, encryption of payload, access restriction to objects.
Please let me know if * you are interested in this library in some way * have comments * know very similar libraries or * are interested in serialization using variadic function templates.
Regards, Siegfried Kettlitz
---
A short introduction on the library:
* "nodes" are the central parts. They contain a factory for objects, store objects and forward "messages" to objects or other nodes in the network and contain a "scheduler" for executing function calls. The nodes are connected by "channels", which forward messages between the two connected nodes. Currently there is a TCP/IPv4 implementation using ASIO and direct forwarding.
* Nodes connected by channels form a network, create and monitor virtual connections between each other. The network map is used to route messages between connected nodes.
* An object has to supply a template for a class name (only if it can be constructed by the factory) and a template for registering the callable functions (also for abstract base classes which can't be constructed).
* A function call is executed asynchronously (wrapper for synchronous call exists). The call to node.call(...) returns a "link_caller"-object. The function to be executed itself can optionally have "link_callee" as the first argument (which is invisible for the caller). The "link_caller"/"link_callee" is used for synchronization (wait for call to begin/finish execution), message forwarding (push/pop strings at caller/callee) and transfer/extraction of the return value.
* Objects of class hierarchies with abstract base classes are possible and handled transparently.
* Objects can be constructed by the factory of a node or an existing (externally constructed) object can be registered by the node.
* Objects don't necessarily need to know (be modified, inherit something etc.) that they are part of the RPC-system and some of their functions are called remotely.
* Objects can be aware that they are constructed by the factory and can, on construction be informed about their identity and get a reference to the node to make remote calls themselves.
* An object is identified by a "reference" consisting of the "node_id" and the "object_id".
* Parameters are passed (serialized, deserialized) by value. All function parameters must have the possibility to be serialized. Modifying referenced parameters won't affect the parameter at the caller. Pointers obviously won't work between nodes not sharing the same memory-address-space.
* Called functions can use "link_callee" to make remote calls themselves, regardless of the object.
* Priorities are assigned to calls and affect message forwarding and scheduling.
* Calls to objects can be transformed into a boost::function using node.bind(...).
* If a called function throws, the exception is forwarded as rpc::exception or a more specialized exception class. The exception is thrown at the caller, when waiting for interaction with the callee (e.g. wait for return value).
* Template functions as targets are possible when specifying the template parameters of the function pointer when registering the function.
* There is no need to use macros at all, although macros could be used to simplify some things.
----------
// Example for creating and calling an object.
// Create a service. boost::asio::io_service node_io_service;
// Create a node. rpc::node node_s(node_io_service, config);
// Register the builder in order to build the server object later. node_s.register_builderdevicelib::chat_server_impl();
// Register the interface of the client to be able to make calls to it. node_s.register_interfacedevicelib::chat_client();
// Create the server object. rpc::reference cs( node_s.get_id(), node_s.allocate_object_templatedevicelib::chat_server_impl() );
// Store the server in name server. // rpc::reference( node_s.get_id(), rpc::oid_simple_object_name_server ) -> A reference to the object_name_server. // rpc::sched_default() -> Default scheduling parameters. // &rpc::simple_object_name_server::store, cs -> function to call. // cs, "chat_server_object" -> function parameters. node_s.wrap_call( rpc::reference( node_s.get_id(), rpc::oid_simple_object_name_server ), rpc::sched_default(), &rpc::simple_object_name_server::store, cs, "chat_server_object" );
----------
// Example for abstract base class. namespace devicelib { class chat_server : public chat_types { public:
virtual void receive( std::string xml_message ) = 0;
virtual user_id_t connect_user( rpc::reference a_client, std::string xml_user_info ) = 0; virtual std::string query_user( user_id_t id ) = 0; virtual void disconnect_user( user_id_t user ) = 0;
virtual void join_room( user_id_t user, room_id_t room ) = 0;
virtual std::list
list_users( ) = 0; virtual std::list list_rooms( ) = 0; }; }
---------------
// Example for registering the interface of the server object.
// The template is declared in namespace rpc, therefore we need to define it there. namespace rpc {
// For the base object. template<> void register_proxy_functionsdevicelib::chat_server( function_call_proxydevicelib::chat_server& ar_proxy ) { // Register the functions of the abstract base object using unique strings. ar_proxy.register_interface( &devicelib::chat_server::connect_user, "&chat_server::connect_user" ); ar_proxy.register_interface( &devicelib::chat_server::query_user, "&chat_server::query_user" ); ar_proxy.register_interface( &devicelib::chat_server::disconnect_user, "&chat_server::disconnect_user" ); ar_proxy.register_interface( &devicelib::chat_server::join_room, "&chat_server::join_room" ); ar_proxy.register_interface( &devicelib::chat_server::list_users, "&chat_server::list_users" ); ar_proxy.register_interface( &devicelib::chat_server::list_rooms, "&chat_server::list_rooms" ); }
// For each implementation. template<> void register_proxy_functionsdevicelib::chat_server_impl( function_call_proxydevicelib::chat_server_impl& ar_proxy ) { // Register the functions of the base object. register_proxy_functionsdevicelib::chat_server( (function_call_proxydevicelib::chat_server&) ar_proxy ); // Make the functions of this object available. ar_proxy.register_interface( &devicelib::chat_server_impl::connect_user, &devicelib::chat_server::connect_user ); ar_proxy.register_interface( &devicelib::chat_server_impl::disconnect_user, &devicelib::chat_server::disconnect_user ); ar_proxy.register_interface( &devicelib::chat_server_impl::join_room, &devicelib::chat_server::join_room ); ar_proxy.register_interface( &devicelib::chat_server_impl::list_users, &devicelib::chat_server::list_users ); ar_proxy.register_interface( &devicelib::chat_server_impl::list_rooms, &devicelib::chat_server::list_rooms ); }
// The implementation can be built, the base object not. template<> std::string class_builder_namedevicelib::chat_server_impl() { // Give it an unique name here. return "chat_server_impl"; } }
-----------------
1: http://www.boostpro.com/vault/index.php?&direction=0&order=&directory=Distributed%20Computing _______________________________________________ Boost-users mailing list Boost-users@lists.boost.org http://lists.boost.org/mailman/listinfo.cgi/boost-users
2009/6/11 Siegfried Kettlitz
Hello boost users and developers,
i'm currently writing a library for making remote procedure calls to C++ objects. The library is based on the variadic templates extension coming with C++0x (already present in GCC 4.3) in conjunction with the boost library (threads, serialization, asio). The aim is to have a library, that is easy to integrate and doesn't require additional tools. Writing it, so far, has thaught me a lot about C++, templates, variadic templates, threading, networking etc..
You might want to take a look at RCF library (
http://www.codeproject.com/KB/threads/RMI_For_Cpp.aspx).
As an example, a simple echo server looks like this:
#include
@Roman:
I took a quick look at Jarl Lindrud RCF library. It is in some ways
similar but very different in others. The separate client/server
concept differs a lot from the node-concept used in my library. I
didn't quite understand how objects are handled and addressed in RCF
and if it is possible to have two objects of the same type at one
server and addressable by one client.
Marshalling of in/out variables is one part that is missing in my
library. This would be nice for synchronous calls, but could be very
bad for asynchronous calls.
Using macros to register the functions may be nice for users, but it
should not be necessary (which it is for RCF). It provides a proxy
class for making the calls to remote functions, which is nice to have.
One thing i don't like about RCF is, that it needs the program-wide
initialization of the framework in contrast to instancing nodes in my
library. Another thing is the dependence on TCP connections, sometimes
you need to connect clients where the IP is not available (e.g. via
RS232 over optical coupling).
After all RCF looks like a more specialized approach compared to my
library, but RCF looks like it's easier to get started with than my
library so far - something that needs improvement.
Do you have experience with using RCF?
@Jens Weller
The type of serialization is selected by a typedef:
#include
Hello,
i'm currently writing a library for making remote procedure calls to C++ objects.
FYI, last year Stjepan spent quite some time on a rpc library, have you had a chance to look of it? ( https://svn.boost.org/trac/boost/browser/sandbox/rpc). I remember he implemented asynchronous calls and use futures<> to implement synchronous calls. If you search last year's email, you may find good discussions about this rpc lib.
I'll have a closer look on what can be merged with "channels" from the boost vault[1], but from the first look it seems that "channels" targets a much more general approach. After the first tries, i realized, that my implemention has to be very specific (e.g. templates only where necessary). So, the design approaches might be too different.
Yes, the Channel lib i am working on has a quite different target. It intends to define / provide the basic primitives for message passing (various name-spaces and dispatchers), and a template framework to compose these primitives and create customized message passing systems for specific applications. And the resulted messaging systems should be quite simple. Channel generated messaging systems are peer-peer systems, not like the client-server model of RPC. When 2 remote channels connect, the ids / names in these 2 channels are exchanged automatically thru Channel's protocol, so threads / objects connected with these channels can do messaging with remote peers transparently. Another big difference is the use of "names" / "ids". In normal rpc systems (or rpc based systems such as Corba), the only purpose of object names/urls or name-server is for "boot-strapping" - obtaining the inital reference to remote server object, after that, names/urls has no use. In RPC systems, the interface of components are the remote methods exposed by server objects (defined in IDL). In Channel, "names/ids" and name-spaces provide the major design scheme / framework for distributed systems. The interface of a component in distributed systems are what "ids" it publish and what "ids" it subscrib to, its physical location doesn't matter much. A component running inside a process "A", attached to a channel and pub/sub some ids, can easily be moved to another process "B" in a different machine, as long as "B" has a channel which connect to "A"'s channel. The component will function as normal as it was in process "A". RPC is a really important in many domains, although i talked mostly about Channel above. Please keep up your good work! Regards Yigong
On Tue, Jun 16, 2009 at 11:38 PM, Yigong Liu
Hello,
i'm currently writing a library for making remote procedure calls to C++ objects.
FYI, last year Stjepan spent quite some time on a rpc library, have you had a chance to look of it? (https://svn.boost.org/trac/boost/browser/sandbox/rpc). I remember he implemented asynchronous calls and use futures<> to implement synchronous calls. If you search last year's email, you may find good discussions about this rpc lib.
I'll have a closer look on what can be merged with "channels" from the boost vault[1], but from the first look it seems that "channels" targets a much more general approach. After the first tries, i realized, that my implemention has to be very specific (e.g. templates only where necessary). So, the design approaches might be too different.
Yes, the Channel lib i am working on has a quite different target. It intends to define / provide the basic primitives for message passing (various name-spaces and dispatchers), and a template framework to compose these primitives and create customized message passing systems for specific applications. And the resulted messaging systems should be quite simple.
Channel generated messaging systems are peer-peer systems, not like the client-server model of RPC. When 2 remote channels connect, the ids / names in these 2 channels are exchanged automatically thru Channel's protocol, so threads / objects connected with these channels can do messaging with remote peers transparently.
Another big difference is the use of "names" / "ids". In normal rpc systems (or rpc based systems such as Corba), the only purpose of object names/urls or name-server is for "boot-strapping" - obtaining the inital reference to remote server object, after that, names/urls has no use. In RPC systems, the interface of components are the remote methods exposed by server objects (defined in IDL). In Channel, "names/ids" and name-spaces provide the major design scheme / framework for distributed systems. The interface of a component in distributed systems are what "ids" it publish and what "ids" it subscrib to, its physical location doesn't matter much. A component running inside a process "A", attached to a channel and pub/sub some ids, can easily be moved to another process "B" in a different machine, as long as "B" has a channel which connect to "A"'s channel. The component will function as normal as it was in process "A".
RPC is a really important in many domains, although i talked mostly about Channel above. Please keep up your good work!
On the open source (but not free in cost for commercial app) networking library RakNet, I created a templated RPC interface for it. It is very efficient, type-safe on both side, and the external interface is more simple then what you have above. Quite literally you just define you function, class member, whatever, then just register it. There is a helper thing to that it returns that you call instead. The tutorial video is at http://www.jenkinssoftware.com/raknet/manual/RPC3Video.htm and if you download it, it is RPC3. I wrote that and donated it to his project. I mostly ripped it out of an old script registration system (that build up the variable conversion operations and called the native C/C++ function as appropriate). It makes heavy use of Boost.Fusion (this was right when Fusion was added to Boost). It made for a wonderful and easy to use system. The above tutorial mostly focuses on features that his old assembly driven RPC system was not capable of, but mine had more features that he never seems to use. For example, when you register a function you can store it in a boost/tr1::function<> object, then if you call that instead it saves a map lookup for the name, and based on the setup you set for it (only call on client systems, only call on server, only call on owner, etc...) it called everything appropriately, very powerful (I really wish he used that feature, I do not like the fact he shows them how to do it dynamically, hence the hashmap lookup... But yea, look at the tutorial video for the basic feature rundown (I did not make that video, Rak'kar did). Another feature mine has that he does not display, if your function has a parameter of Raknet::RPC3* then it auto-fills that in with the RPC3 registration object, letting you get detailed information about your call like if it is a remote call, local call, etc... It also check for various class hierarchies, such as if a NetworkIDObject is a subclass of any class pointer passed in, it will use the corresponding registered NetworkIDObject on the remote system and so forth. Raknet::Bitstream is a bitstream as you would expect (I really want a bitstream like Raknet::Bitstream in boost, does anyone know if one already exists? the only ones I have found do not have the capabilities), and I had him add the operator
/<< functions so streaming can be overloaded.
Either way, he changed the code slightly, added a few more default overloads, etc... since I gave it to him a long while back (back in Boost 1.35's time period, 1.35 was new), but take a look at the RPC3 code and see how it does it, actually very simple, very powerful, type-safe, overloadability, etc... I would be quite happy if my code ended up in Boost. But, thanks to fusion and such, does not require variadic templated (although tr1::function is the main restriction, if it uses variadic templates then there is no real limit to my system then).
On the open source (but not free in cost for commercial app)
Not being free to use it in a commercial application is kind of a no-go for many areas like in boost i think. Your library therefore needs to be independent of RakNet.
networking library RakNet, I created a templated RPC interface for it. It is very efficient, type-safe on both side, and the external interface is more simple then what you have above. Quite literally you just define you function, class member, whatever, then just register it.
I looked at the example in RakNet but didn't look at the source yet. How is this call made type-safe? rpc3Inst.CallCPP("&C::ClassMemberFunc", GetNetworkID(), a1,a2,c1,c2,RakNet::_RPC3::Deref(d1),d2,bs1,bs2,rpcFromNetwork); How do you avoid naming collisions with other namespaces?
There is a helper thing to that it returns that you call instead. The tutorial video is at http://www.jenkinssoftware.com/raknet/manuhttp://www.jenkinssoftware.com/raknet/manual/RPC3Video.htm al/RPC3Video.htmhttp://www.jenkinssoftware.com/raknet/manual/RPC3Video.htmand if you download it, it is RPC3. I wrote that and donated it to his project. I mostly ripped it out of an old script registration system (that build up the variable conversion operations and called the native C/C++ function as appropriate). It makes heavy use of Boost.Fusion (this was right when Fusion was added to Boost). It made for a wonderful and easy to use system. The above tutorial mostly focuses on features that his old assembly driven RPC system was not capable of, but mine had more features that he never seems to use. For example, when you register a function you can store it in a boost/tr1::function<> object, then if you call that instead it saves a map lookup for the name, and based on the setup you set for it (only call on client systems, only call on server, only call on owner, etc...) it called everything appropriately, very powerful (I really wish he used that feature, I do not like the fact he shows them how to do it dynamically, hence the hashmap lookup...
It looks like there's one object registration for the whole network in RPC3, while in my library every node has its own registrations because nodes can join and leave the network at every point in time. There isn't even "one network" because there are only real and virtual peer-to-peer-connections between nodes.
But yea, look at the tutorial video for the basic feature rundown (I did not make that video, Rak'kar did). Another feature mine has that he does not display, if your function has a parameter of Raknet::RPC3* then it auto-fills that in with the RPC3 registration object, letting you get detailed information about your call like if it is a remote call, local call, etc...
This is analogous to the optional "link_callee&" reference in my library and it's really useful for functions aware of the remote calls.
It also check for various class hierarchies, such as if a NetworkIDObject is a subclass of any class pointer passed in, it will use the corresponding registered NetworkIDObject on the remote system and so forth. Raknet::Bitstream is a bitstream as you would expect (I really want a bitstream like Raknet::Bitstream in boost, does anyone know if one already exists? the only ones I have found do not have the capabilities), and I had him add the operator
/<< functions so streaming can be overloaded.
That's one advantage over my library, because so far, nothing other is done with the parameters but serializing/deserializing them to call the function. It's not aware of networking capable objects. And i haven't checked, how it would handle pointers to objects.
Either way, he changed the code slightly, added a few more default overloads, etc... since I gave it to him a long while back (back in Boost 1.35's time period, 1.35 was new), but take a look at the RPC3 code and see how it does it, actually very simple, very powerful, type-safe, overloadability, etc... I would be quite happy if my code ended up in Boost.
Is your library independent of RakNet? Can you put your code under the boost-license? When reading the mailing list, the process of putting a library into boost is rather time-consuming and can take years until everything works on all required compilers is well documented and in in harmony with the other boost libraries. From that point on, it requires constant attention to keep up with new compilers and changes in other boost libraries.
But, thanks to fusion and such, does not require variadic templated (although tr1::function is the main restriction, if it uses variadic templates then there is no real limit to my system then).
Not being forced to require variadic templates indeed is a big plus for the next 3 years until all common complilers understand them correctly. Until then i'll be mostly restricted to GCC. Thanks for you interesting reply, Siegfried
On Fri, Jun 19, 2009 at 12:36 AM, Siegfried
Kettlitz
On the open source (but not free in cost for commercial app)
Not being free to use it in a commercial application is kind of a no-go for many areas like in boost i think. Your library therefore needs to be independent of RakNet.
Only in that I gave him that code to use, I still have my own code. I
altered it from a scripting language binding I created, it
automatically built the scripting links necessary just by passing the
function (kind of like Boost::Python, but a little more powerful).
The code itself to do the calls, if you ignore all the RakNet stuff
and the extra overloaded operators to handle things like the
NetworkIDObjects and so forth is actually very small, might fit on a
printed page actually.
So yes, I can give you code that is independent of RakNet, the code is
very simple though, but I can walk you through it if you wish, just
makes use of Boost.Fusion::invoke.
On Fri, Jun 19, 2009 at 12:36 AM, Siegfried
Kettlitz
networking library RakNet, I created a templated RPC interface for it. It is very efficient, type-safe on both side, and the external interface is more simple then what you have above. Quite literally you just define you function, class member, whatever, then just register it.
I looked at the example in RakNet but didn't look at the source yet.
How is this call made type-safe?
rpc3Inst.CallCPP("&C::ClassMemberFunc", GetNetworkID(), a1,a2,c1,c2,RakNet::_RPC3::Deref(d1),d2,bs1,bs2,rpcFromNetwork);
How do you avoid naming collisions with other namespaces?
There is a helper thing to that it returns that you call instead. The tutorial video is at http://www.jenkinssoftware.com/raknet/manual/RPC3Video.htm and if you download it, it is RPC3. I wrote that and donated it to his project. I mostly ripped it out of an old script registration system (that build up the variable conversion operations and called the native C/C++ function as appropriate). It makes heavy use of Boost.Fusion (this was right when Fusion was added to Boost). It made for a wonderful and easy to use system. The above tutorial mostly focuses on features that his old assembly driven RPC system was not capable of, but mine had more features that he never seems to use. For example, when you register a function you can store it in a boost/tr1::function<> object, then if you call that instead it saves a map lookup for the name, and based on the setup you set for it (only call on client systems, only call on server, only call on owner, etc...) it called everything appropriately, very powerful (I really wish he used that feature, I do not like the fact he shows them how to do it dynamically, hence the hashmap lookup...
It looks like there's one object registration for the whole network in RPC3, while in my library every node has its own registrations because nodes can join and leave the network at every point in time. There isn't even "one network" because there are only real and virtual peer-to-peer-connections between nodes. There is not one registration in RakNet. He wanted the macros introduced for ease-of-use, but if you do not use them then you can even dynamically register and unregister things over the lifetime of
When you register the call, if you have identically named functions
then just register it directly instead of using the macro, the only
thing the macro does is basically this (Rak'kar added the
'cpp/c/etc...' names, my system actually supports anything directly
with no difference, even functors, yes you can register functors):
#define REGISTER_RPC_CALL(func, rpc) rpc->RegisterCall(func, #func, 0);
Just make that call yourself and put a namespace specific name for the
#func param. The flags I initially added to handle a vastly more
powerful system, which he never ended up using... >.<
The RegisterCall (not its actual name, but I do not have the code
available right now, at work) is just a templated function for the
func param (the #func is an std::string&) of boost::function, then
through use of boost::function_types and boost::fusion and an external
struct it verifies it returns void, it recurses through all the types
of that function parameters and creates a templated struct for each
one (that is optimized out in release mode, I checked the machine
code). The templated structs can be overridden for detailed behavior
of specific types (like for pointers and references and such), but the
general case just operator>>/<<() the data with the Raknet::Bitstream
(or whatever you will use), which lets the user override how it
serializes the data as well.
If is very simple code though, as I recall, one of the examples in
boost::fusion for invoke is very close to what I did, might look at
that.
On Fri, Jun 19, 2009 at 12:36 AM, Siegfried
Kettlitz
But yea, look at the tutorial video for the basic feature rundown (I did not make that video, Rak'kar did). Another feature mine has that he does not display, if your function has a parameter of Raknet::RPC3* then it auto-fills that in with the RPC3 registration object, letting you get detailed information about your call like if it is a remote call, local call, etc...
This is analogous to the optional "link_callee&" reference in my library and it's really useful for functions aware of the remote calls. Yes, provides a noticable speed boost over an unordered_map lookup, I really wish he advertised that feature of it, but the video is more geared toward new programmers rather then real programmers.
On Fri, Jun 19, 2009 at 12:36 AM, Siegfried
Kettlitz
It also check for various class hierarchies, such as if a NetworkIDObject is a subclass of any class pointer passed in, it will use the corresponding registered NetworkIDObject on the remote system and so forth. Raknet::Bitstream is a bitstream as you would expect (I really want a bitstream like Raknet::Bitstream in boost, does anyone know if one already exists? the only ones I have found do not have the capabilities), and I had him add the operator
/<< functions so streaming can be overloaded.
That's one advantage over my library, because so far, nothing other is done with the parameters but serializing/deserializing them to call the function. It's not aware of networking capable objects. And i haven't checked, how it would handle pointers to objects. That is just one of the struct overloads (like with pointers and such), it uses enable_if to enable and disable various specs depending on if the class inherits from NetworkIDObject, can of course be expanded. NetworkIDObject is just a 64-bit integer that matches one on another system for matching serialization calls between the objects in RakNet.
On Fri, Jun 19, 2009 at 12:36 AM, Siegfried
Kettlitz
Either way, he changed the code slightly, added a few more default overloads, etc... since I gave it to him a long while back (back in Boost 1.35's time period, 1.35 was new), but take a look at the RPC3 code and see how it does it, actually very simple, very powerful, type-safe, overloadability, etc... I would be quite happy if my code ended up in Boost.
Is your library independent of RakNet? Can you put your code under the boost-license?
When reading the mailing list, the process of putting a library into boost is rather time-consuming and can take years until everything works on all required compilers is well documented and in in harmony with the other boost libraries. From that point on, it requires constant attention to keep up with new compilers and changes in other boost libraries.
The code it came from depends on raknet, but the code itself only
requires changes on maybe 2 or 3 lines to support the serialization on
a different system type, and the code it came from, my script system,
is free for whatever use (just not released yet, but I would be quite
happy to generate code for your system). What network archetecture
would I build it around, Boost::ASIO or something, or perhaps it would
be best to build it based on concepts so it is more generic? I doubt
it is complete enough to ever be considered a standalone library in
Boost, maybe an example for Boost.ASIO or Boost.Fusion or something.
On Fri, Jun 19, 2009 at 12:36 AM, Siegfried
Kettlitz
But, thanks to fusion and such, does not require variadic templated (although tr1::function is the main restriction, if it uses variadic templates then there is no real limit to my system then).
Not being forced to require variadic templates indeed is a big plus for the next 3 years until all common complilers understand them correctly. Until then i'll be mostly restricted to GCC.
Yea, such things did not exist a couple years ago when I made this, so it definitely works without it. And for note, my main compiler is Visual Studio 2005, so I could not use yours either. :)
Hi Yigong,
FYI, last year Stjepan spent quite some time on a rpc library, have you had a chance to look of it? ( https://svn.boost.org/trac/boost/browser/sandbox/rpc). I remember he implemented asynchronous calls and use futures<> to implement synchronous calls. If you search last year's email, you may find good discussions about this rpc lib.
This is for calls to functions and not member-functions. I shouldn't call my
library doing Remote Procedure Calls, but Remote Member Calls.
The approach using futures is one thing, i'd like to implement, too, since
everything needed for that is already in place.
One thing i don't like about that library is, that you need to provide the
interface explicitly multiple times:
reg.set
Yes, the Channel lib i am working on has a quite different target. It intends to define / provide the basic primitives for message passing (various name-spaces and dispatchers), and a template framework to compose these primitives and create customized message passing systems for specific applications. And the resulted messaging systems should be quite simple.
Channel generated messaging systems are peer-peer systems, not like the client-server model of RPC. When 2 remote channels connect, the ids / names in these 2 channels are exchanged automatically thru Channel's protocol, so threads / objects connected with these channels can do messaging with remote peers transparently.
Can more than two channels connect? If so, consider A connected to B, B connected to C. Does A see C and can communicate with it? Another big difference is the use of "names" / "ids". In normal rpc systems
(or rpc based systems such as Corba), the only purpose of object names/urls or name-server is for "boot-strapping" - obtaining the inital reference to remote server object, after that, names/urls has no use. In RPC systems, the interface of components are the remote methods exposed by server objects (defined in IDL). In Channel, "names/ids" and name-spaces provide the major design scheme / framework for distributed systems. The interface of a component in distributed systems are what "ids" it publish and what "ids" it subscrib to, its physical location doesn't matter much. A component running inside a process "A", attached to a channel and pub/sub some ids, can easily be moved to another process "B" in a different machine, as long as "B" has a channel which connect to "A"'s channel. The component will function as normal as it was in process "A".
This sounds very much like Plan9. Can the namespace-model be built upon a normal RPC-library without much effort? I assume, that something like a dynamic name->id resolver added to the RPC-library provides just what you want. Instead of native references, the objects need to be addressed by the "names". When an object changes its id, the resolver publishes that information and the calls go to the "new" target. Although defining abstract interfaces for such an implementation is still a good idea and in the long run better than having multiple implementations with incompatible interfaces.
RPC is a really important in many domains, although i talked mostly about Channel above. Please keep up your good work!
Thanks, I will. Siegfried
Seigfried, Around the same time your message was posted, I implemented an RPC library on top of C++98 using boost serialization that is communications medium neutral (data can be sent/received with a simple subclass) and supports: * callback and blocking calls * deferred completion * exceptions It also requires no macros by the user. RPC function signatures/prototypes are defined by a simple instance of a template. boost::bind is used to implement an RPC. There would have to be some interest in it before HP would let me release it. Thanks, Eric Siegfried Kettlitz wrote:
Hello boost users and developers,
i'm currently writing a library for making remote procedure calls to C++ objects. The library is based on the variadic templates extension coming with C++0x (already present in GCC 4.3) in conjunction with the boost library (threads, serialization, asio). The aim is to have a library, that is easy to integrate and doesn't require additional tools. Writing it, so far, has thaught me a lot about C++, templates, variadic templates, threading, networking etc..
I post this message, because i think that this library is quite handy in situations, where a developer already uses boost libraries and wants to add some networking functionality to his/her project and the usage of variadic templates instead of additional tools is quite new. One example would be remote monitoring an application by adding a "std::string get_status()"-function to some object. Implemented as example is a chat-system with a server and clients interacting by remote calls. This example is tested with a x64-linux and win32 using the portable boost-text-serialization. Multiple clients can connect to the server via the TCP/IP-acceptor (and disconnect without crashing the system). Implemented is a console-client for text input and display and a morse client, that interacts via a digital output device (relais card), which is connected via a serial port. The serial port and the digital output driver are both also connected via the rpc-system and can be physically located on separate computers.
At this point, i have the basic bits together and a working rpc-library, that needs to be cleaned, refined and extended to match the practical needs. This means, that some programs and libraries have to be written, which use the rpc-library and show its bugs and limitations. The downside of it is, that the interface of the rpc-library may change one more time until it can be considered stable. Therefore it would not be good to test it widely in libraries posing similar requirements, but to test it with a variety of possible applications and provide the implementations as examples and (regression-/unit-)tests.
- One library for Measurement/Automation is already in development. - The chat-server/client exists. - A transaction based game server is under consideration. - Some distributed parallel computing application with dynamic joining/leaving of clients.
Can you give me some more ideas of different applications, which are easy to implement but yet useful? What would you like to have implemented and accessible by an rpc-library?
One speed measurement using local (shared memory) channels resulted in 20000 calls/s and 60000 messages/s on a 1.8GHz Opteron. Do you know how this compares to other implementations on similar hardware?
The TCP/IP-channel implementation using ASIO resulted in only about 100 synchronous calls/second regardless of the payload and almost no CPU load, which suggests that ASIO somewhere "sleeps" on linux systems for a few milliseconds. This would be fixed in some future implementation. Using asynchronous calls (1000 open calls) again leads to about 10000 calls/s.
I'll have a closer look on what can be merged with "channels" from the boost vault[1], but from the first look it seems that "channels" targets a much more general approach. After the first tries, i realized, that my implemention has to be very specific (e.g. templates only where necessary). So, the design approaches might be too different.
After the refinement, cleanup, some testing and documenting, i'd like to publish the rpc-library (and libraries based on it) under GPL- or boost-license. Then you'll hear from me again. In future i'll probably work 2-8 hours per week on the library and use it in some projects, but there's plenty of work left to do in various regions. This will leave space for other developers who might want to use and modify it to fit their needs. Some of the harder parts that would need implementation are redundant parallel connections and advanced routing, authentication of nodes, signing of messages, encryption of payload, access restriction to objects.
Please let me know if * you are interested in this library in some way * have comments * know very similar libraries or * are interested in serialization using variadic function templates.
Regards, Siegfried Kettlitz
---
A short introduction on the library:
* "nodes" are the central parts. They contain a factory for objects, store objects and forward "messages" to objects or other nodes in the network and contain a "scheduler" for executing function calls. The nodes are connected by "channels", which forward messages between the two connected nodes. Currently there is a TCP/IPv4 implementation using ASIO and direct forwarding.
* Nodes connected by channels form a network, create and monitor virtual connections between each other. The network map is used to route messages between connected nodes.
* An object has to supply a template for a class name (only if it can be constructed by the factory) and a template for registering the callable functions (also for abstract base classes which can't be constructed).
* A function call is executed asynchronously (wrapper for synchronous call exists). The call to node.call(...) returns a "link_caller"-object. The function to be executed itself can optionally have "link_callee" as the first argument (which is invisible for the caller). The "link_caller"/"link_callee" is used for synchronization (wait for call to begin/finish execution), message forwarding (push/pop strings at caller/callee) and transfer/extraction of the return value.
* Objects of class hierarchies with abstract base classes are possible and handled transparently.
* Objects can be constructed by the factory of a node or an existing (externally constructed) object can be registered by the node.
* Objects don't necessarily need to know (be modified, inherit something etc.) that they are part of the RPC-system and some of their functions are called remotely.
* Objects can be aware that they are constructed by the factory and can, on construction be informed about their identity and get a reference to the node to make remote calls themselves.
* An object is identified by a "reference" consisting of the "node_id" and the "object_id".
* Parameters are passed (serialized, deserialized) by value. All function parameters must have the possibility to be serialized. Modifying referenced parameters won't affect the parameter at the caller. Pointers obviously won't work between nodes not sharing the same memory-address-space.
* Called functions can use "link_callee" to make remote calls themselves, regardless of the object.
* Priorities are assigned to calls and affect message forwarding and scheduling.
* Calls to objects can be transformed into a boost::function using node.bind(...).
* If a called function throws, the exception is forwarded as rpc::exception or a more specialized exception class. The exception is thrown at the caller, when waiting for interaction with the callee (e.g. wait for return value).
* Template functions as targets are possible when specifying the template parameters of the function pointer when registering the function.
* There is no need to use macros at all, although macros could be used to simplify some things.
----------
// Example for creating and calling an object.
// Create a service. boost::asio::io_service node_io_service;
// Create a node. rpc::node node_s(node_io_service, config);
// Register the builder in order to build the server object later. node_s.register_builderdevicelib::chat_server_impl();
// Register the interface of the client to be able to make calls to it. node_s.register_interfacedevicelib::chat_client();
// Create the server object. rpc::reference cs( node_s.get_id(), node_s.allocate_object_templatedevicelib::chat_server_impl() );
// Store the server in name server. // rpc::reference( node_s.get_id(), rpc::oid_simple_object_name_server ) -> A reference to the object_name_server. // rpc::sched_default() -> Default scheduling parameters. // &rpc::simple_object_name_server::store, cs -> function to call. // cs, "chat_server_object" -> function parameters. node_s.wrap_call( rpc::reference( node_s.get_id(), rpc::oid_simple_object_name_server ), rpc::sched_default(), &rpc::simple_object_name_server::store, cs, "chat_server_object" );
----------
// Example for abstract base class. namespace devicelib { class chat_server : public chat_types { public:
virtual void receive( std::string xml_message ) = 0;
virtual user_id_t connect_user( rpc::reference a_client, std::string xml_user_info ) = 0; virtual std::string query_user( user_id_t id ) = 0; virtual void disconnect_user( user_id_t user ) = 0;
virtual void join_room( user_id_t user, room_id_t room ) = 0;
virtual std::list
list_users( ) = 0; virtual std::list list_rooms( ) = 0; }; }
---------------
// Example for registering the interface of the server object.
// The template is declared in namespace rpc, therefore we need to define it there. namespace rpc {
// For the base object. template<> void register_proxy_functionsdevicelib::chat_server( function_call_proxydevicelib::chat_server& ar_proxy ) { // Register the functions of the abstract base object using unique strings. ar_proxy.register_interface( &devicelib::chat_server::connect_user, "&chat_server::connect_user" ); ar_proxy.register_interface( &devicelib::chat_server::query_user, "&chat_server::query_user" ); ar_proxy.register_interface( &devicelib::chat_server::disconnect_user, "&chat_server::disconnect_user" ); ar_proxy.register_interface( &devicelib::chat_server::join_room, "&chat_server::join_room" ); ar_proxy.register_interface( &devicelib::chat_server::list_users, "&chat_server::list_users" ); ar_proxy.register_interface( &devicelib::chat_server::list_rooms, "&chat_server::list_rooms" ); }
// For each implementation. template<> void register_proxy_functionsdevicelib::chat_server_impl( function_call_proxydevicelib::chat_server_impl& ar_proxy ) { // Register the functions of the base object. register_proxy_functionsdevicelib::chat_server( (function_call_proxydevicelib::chat_server&) ar_proxy ); // Make the functions of this object available. ar_proxy.register_interface( &devicelib::chat_server_impl::connect_user, &devicelib::chat_server::connect_user ); ar_proxy.register_interface( &devicelib::chat_server_impl::disconnect_user, &devicelib::chat_server::disconnect_user ); ar_proxy.register_interface( &devicelib::chat_server_impl::join_room, &devicelib::chat_server::join_room ); ar_proxy.register_interface( &devicelib::chat_server_impl::list_users, &devicelib::chat_server::list_users ); ar_proxy.register_interface( &devicelib::chat_server_impl::list_rooms, &devicelib::chat_server::list_rooms ); }
// The implementation can be built, the base object not. template<> std::string class_builder_namedevicelib::chat_server_impl() { // Give it an unique name here. return "chat_server_impl"; } }
-----------------
1: http://www.boostpro.com/vault/index.php?&direction=0&order=&directory=Distributed%20Computing
participants (8)
-
Damien Hocking -
Eric Woodruff -
Jens Weller -
Ovanes Markarian -
OvermindDL1 -
Roman Perepelitsa -
Siegfried Kettlitz -
Yigong Liu