Hello, I have worked some on the RPC/marshal implementation. FWIW, I'm attaching the new code - it builds on MSVC 8.0 and apple's branch of GCC 4.0.1, BUT it doesn't run too well. The network code is thrown together and there are network/race condition issues. Also, the heap gets messed up along the way even when the RPCs get executed correctly. *sigh* Anyway, before I go into working this out bottom up and adding docs, it would be better for me to get a better idea of what the end result should be. So, here is how the framework functions now. 1. The server-side components: 1.1. The registry - this contains a collection of functions mapped by key: // make a registry where the functions are keyed by int: marshal::registry<int> reg; 1.2. The function - each function gets added to the registry so it can be accessed: int add2( int i, int j ) { return i+j; } ... reg.set<int (int, int)>(2, add2); // register function 1.3. registry_server - this will service calls from an individual client, and is associated with a particular registry. // Make a server that uses a simple connection protocol marshal::server<marshal::registry<int>, marshal::simple_acceptor> server(reg, 1097); The simple_acceptor just grants a connection to anyone that asks for it. The Acceptor is a template parameter, so different connection/authentication protocols could be used (although they should probably be in a different library, since they are not necessarily specific to RPC). 1.4. The server - the server will accept connections, and make a new registry_server object which will serve the new socket 2. Client-side components 2.1. The client marshal::client<marshal::registry<int>, marshal::simple_connector> client( ip::tcp::endpoint(ip::address::from_string("127.0.0.1"), 1097)); Negotiates a connection to the server and can service calls to the registry on the server side. 2.2. The call marshal::call<int (int, int)> call_5_6(5,6); A call now stores the parameters only, and (possibly) storage for the return value of the function. 2.3. Making the call client(2, call_5_6, marshal::call_options( marshal::call_options::return_only, marshal::call_options::async)); A call is given to the client, with a function id, the parameters stored in the marshal::call class, and call options. Call options specify: - what to marshal back (nothing, just an acknowledgement that the function has completed, just the return value, or (not implemented yet) values that can be returned through parameters) - sync (don't return until the function call has completed) vs. async (return immediately) 2.4 The results // it was an async call, wait until it has completed: while(!call_5_6.has_completed()) {} // now we can get the return value because we asked for it BOOST_CHECK(call_5_6.return_val() == 11); The test file included in the code has only a few more examples. My main questions are: * right now, the entire server-side code is running in only one thread. Should there be one thread per client? One thread per function call? Is there a "best" solution or should there be options on this? * the above schemes separates the call into - the id - the parameters - the options (what to marshal back, sync vs. async, (exception reporting to come)) It is possible that the implementation of a sync call will be substantially different then an async call (for example, an async call might use futures to return the result). Should the sync/async choice be placed with the marshal::call class (i.e., make a marshal::async_call and marshal::sync_call). I can see both advantages and disadvantages to this. * right now, the network protocol is rather clumsy - there's one or two header packets and a marshal packet in each direction (and some acknowledgement packets that I threw in to try to fix things but it didn't seem to help much)... The advantage of this approach is that the header can say how big the marshal packet is (which can vary widely, so any preallocated buffer might be too small), but changing things so that there is only one packet would probably simplify things a whole lot. I'm still getting familiar with Boost.Asio... if anyone has any thoughts on what to do I'd appreciate it. OK, that's all for now... when I grab some time I'll try to document this thing a little better and hopefully fix it in the process. Please let me know what you think... Thanks, Stjepan On 4/19/07, Scott Woods <scottw@qbik.com> wrote:
----- Original Message ----- From: "Stjepan Rajko" <stipe@asu.edu> To: <boost@lists.boost.org> Sent: Thursday, April 19, 2007 10:53 AM Subject: Re: [boost] Marshalling library
The scope of this list get daunting quickly. What is the scope for you guys?
My personal needs are pretty basic - just making a remote call and getting the results back would cut it. In terms of designing a marshalling framework, I'd propose separating out as much functionality from threading, network, shared memory, and other issues, and sticking to the actual function call and execution (for starts).
What I'm hoping is that things can be designed so that other Boost libraries can transparently address these issues through existing functionality. For example, the readable/fast encodings can be used by allowing the choice of either binary or xml (or text) serialization archives offered by Boost.Serialization. A secure channel can be established by using an SSL asio connection, etc.
How's this for a 1st cut breakdown;
* asynchronous socket manager (definitely a thread or threads, might be in asio) * asynchronous RPC manager (probably a thread) with a sync interface for application access * synchronous application thread(s)
Something like this would deliver what you are after, if in a more cumbersome way than what you might have been thinking. Maybe some magic is created if it also caters to needs such as the following;
* deriving from the base socket manager to create an optimally fast FTP server * ability to communicate with two RPC servers, one running a high-speed proprietary encoding (Boost!) to a local server the other running good old Sun RPC (is that XDR encoding?) to some remote legacy box, the location and associated encoding being transparent to the caller. * ability to switch the local server encoding between fast and readable (i.e. debug vs release) * facility to develop a monitoring and control tool for a socket manager. * ability to instantiate both a base socket manager and the dedicated FTP server in the one executable, each one taking ownership of an accessible interface or port range.
While these requirements might seem contrived, the circumstances are not so unrealistic. If you tackle the intiial goal (RPC) but deliver a collection of types and objects that provides for the needs above then I really think you would have something. Its the decomposition into the useful pieces; that's the intractable bit. If you get it right then its easy to reshuffle into new arrangements such as those above.
Hope that explains my position a bit better. The solution is not obvious to me and thats probably been apparent in my messages.
Cheers.
_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost