Hi Johan, On 4/23/07, Johan Nilsson <r.johan.nilsson@gmail.com> wrote:

Stjepan Rajko wrote:

...

I have uploaded some documentation and code of a prototype RPC / marshal library:

Have you seen this one:

http://www.codeproject.com/threads/Rcf_Ipc_For_Cpp.asp

Thanks for the link! I e-mailed the author but he's too busy to work together on this... Still, its good to have his implementation to learn from. Stjepan

Hi Jeff, On 4/24/07, Jeff Garland <jeff@crystalclearsoftware.com> wrote:

Stjepan Rajko wrote:

...

In addition to what's listed at the website, I'm wondering what the proper way of returning the function call results would be... For example, if the function is void something(int &x), should the modified "x" value on the remote computer appear in the local x after the remote call returns?

Just to add some perspective, 'full RPC' systems typically support this way of returning values. In CORBA, parameters are characterized as 'in', 'out' or 'in-out' in the IDL method descriptions.

In the case when a parameter was specified as an out parameter, or presumed to be an out parameter due to its type, I wasn't sure whether the returned value should go directly into the passed argument, or stored elsewhere. For example, int i=1; remote_call inc_call(inc_function_id, i); rpc_client.make_remote_call(inc_call); // alternative one: now i == 2 // alternative two: now my_call.arg1() == 2 but i==1 Alternative one is a lot cleaner and intuitive, and it doesn't require a lasting call object (i.e., it makes it possible to make the remote call available through a local function that behaves as if the remote function was local: something like remote_inc_call(i) which performs rpc_client.make_remote_call(remote_call(inc_function_id, i)). In retrospect, I don't think there are any good reasons to have alternative two available (at least for sync calls), but I might be wrong. The user can always do int j = i; remote_inc_call(j); .. if they wanted the i value to stay unchanged. So far, I've implemented "alternative one" for sync calls and "alternative two" for async calls. Regarding the in/out/inout specification, I have the following thoughts: * The server can specify which parameters of the function are used which way (in/out/inout). * By default, every parameter is "in" and non-const references are "inout" * The client can override the specified behavior in the following ways: * for an "out" parameter, the client can specify that the output value should not be marshaled back (if the client is not interested in it, to save on the communication, and/or to prevent the old value to be overwritten) * for an "in" parameter, the client can specify that the server should instantiate a default object of the type (if possible), rather than marshaling the default value over the network.

...

I can see that as being reasonable if the RPC is synchronous, but if

Well, not necessarily.

Are you referring to the possibility that the parameter might not be specified as an out parameter, or do you have something else in mind?

...

it is asynchronous maybe something like a future would be a good way of providing the modified value? (speaking of, could someone suggest a futures implementation out of the ones that were floating around a while back?)

http://lists.boost.org/Archives/boost/2007/04/119927.php

Thanks, I downloaded it and will start playing with it for async calls.

If you want to study some past experience, there's a wealth of literature on the designs and tradeoffs. Just a couple examples:

st-www.cs.uiuc.edu/~hanmer/PLoP-97/Proceedings/ritosilva.pdf www.cin.ufpe.br/~phmb/papers/DistributedAdaptersPatternPLOP2001.pdf

I checked them out - thanks for the links!

I have a one other comment for the moment - in the doc you say:

The entire server-side code is running in only one thread. This is probably not good. Should there be one thread per client? One thread per function call? Is there a "best" solution or should there be options on this?

There's not a 'simple' best answer to this. A single thread might be perfectly fine for something that executes a fast function and doesn't serve many clients at the same time (say calculate the current time). Something that needs to execute a function that performs significant computation, thus taking substantial time, needs a different strategy. It might spawn a sub-process or a thread to do the actual work allowing the main thread to wait for and process other inbound connections and requests. A typical strategy for problems that require scalability is to use a thread pool. At any given moment one thread from the pool is waiting for any new i/o on the network -- when it is received that thread begins processing the request and will process it to completion. At the start of request processing another thread takes over waiting for network i/o. This approach allows for minimal context switching w.r.t to processing a request and can be tuned to the number of processors actually available to handle requests and the nature of the processing. Usually the number of threads in this sort of scheme is significantly less than the number of simultaneous clients. Anyway, the 'thread per client' approach is inherently not scalable...which is fine -- as long as you don't need scalability.

Anyway, it's an area of some significant design depth -- and one for which boost doesn't provide all the facilities needed. We don't have the thread pool or thread-safe queue implementations that might be needed in some of the various strategies you might desire.

Great thoughts... Along the lines of a thread pool, asio does provide the facility that would allow me to give it multiple threads as working threads, and it's behavior would I think match the one you describe - a received packet would go to an idle thread, which would then stay busy until the call was completed and returned. Perhaps I can try going this route. Thanks for your thoughts, much appreciated! Stjepan

Stjepan Rajko wrote:

...

...

I can see that as being reasonable if the RPC is synchronous, but if Well, not necessarily.

Are you referring to the possibility that the parameter might not be specified as an out parameter, or do you have something else in mind?

No, my general point really is that the whole idea of 'making an RPC appear like a local function call' is dangerous in my view. There are just too many differences between remote calls and local calls that the programmer ultimately needs to know about. For example, how does the rpc endpoint addressed, what if the endpoint refuses connection, and (specifically related to synchronous behavior) how long is my process blocked while waiting for a response that may never come or may be delayed for several seconds because it takes 3 satellite hops before the result returns? When you make a function call in a normal program you don't think about any of these factors. When you make an RPC, you should be thinking about these things. Similarly, network access is fundamentally asynchronous, so why not program it that way?

Great thoughts... Along the lines of a thread pool, asio does provide the facility that would allow me to give it multiple threads as working threads, and it's behavior would I think match the one you describe - a received packet would go to an idle thread, which would then stay busy until the call was completed and returned. Perhaps I can try going this route.

Which facility is that exactly? Jeff

Stjepan Rajko wrote:

On 4/29/07, Jeff Garland <jeff@crystalclearsoftware.com> wrote:

...

Stjepan Rajko wrote:

...

...

...

I can see that as being reasonable if the RPC is synchronous, but if Well, not necessarily.

Are you referring to the possibility that the parameter might not be specified as an out parameter, or do you have something else in mind? No, my general point really is that the whole idea of 'making an RPC appear like a local function call' is dangerous in my view. There are just too many differences between remote calls and local calls that the programmer ultimately needs to know about. For example, how does the rpc endpoint addressed, what if the endpoint refuses connection, and (specifically related to synchronous behavior) how long is my process blocked while waiting for a response that may never come or may be delayed for several seconds because it takes 3 satellite hops before the result returns? When you make a function call in a normal program you don't think about any of these factors. When you make an RPC, you should be thinking about these things. Similarly, network access is fundamentally asynchronous, so why not program it that way?

I think that there are circumstances in which adopting the simplicity of simulating a local function is worthwhile (you mentioned some in your previous message, e.g. function calls which are quickly evaluated, especially if the network is just a few computers sitting

But you've already fallen into the trap. Just because the computers are next to each other doesn't mean you are going to get an immediate answer. For example, one of the computers you are communicating with might be really busy doing something (swapping perhaps if it's overloaded). It might be so busy the process you are communicating with can't service it's TCP queue. Data starts stacking up in the queue of the OS. The network is fine, no errors are happening, but your simple 'doit' function is now blocked for minutes. It's completely unexpected. In the meantime your application appears to be 'hung' -- it's waiting for a response that isn't coming and instead of 1 machine behaving badly it looks like there's 2. Ok, so you're a smart guy, and now you provide a timeout option on the RPC function so that when it's called you know it will return within 5 seconds (of course you actually expect a response in milliseconds) if there's no response. Great, but now you've blown the cover on this 'synchronous call' because a timeout and timeout error isn't something required on a normal function call. And at this point, is it really that much harder to make an async call and wait for either a timeout or a response? Basically, I'm pushing that programmers should start thinking of async first and sync second because you're going to wind up there most of the time anyway. The sync approach is so inherently limited that it's almost always flawed to use it.

next to each other connected over a local switch), and like you point out, there are circumstances in which ignoring the reality of network communication can be both dangerous and wasteful. FWIW, I think both approaches should be supported, as long as the behavior of the RPC is well designed and documented for both the sync case and the async case.

Well, the worry is that folks tend to gravitate toward the apparently simple solution in contexts where it doesn't work and then get surprised later when these considerations that 'weren't obvious' start cropping up. So most of the time I'd prefer to encourage people to think about the design of what they are doing.

...

...

Great thoughts... Along the lines of a thread pool, asio does provide the facility that would allow me to give it multiple threads as working threads, and it's behavior would I think match the one you describe - a received packet would go to an idle thread, which would then stay busy until the call was completed and returned. Perhaps I can try going this route. Which facility is that exactly?

Documentation of void boost::asio::io_service::run():

"Run the io_service's event processing loop.

The run() function blocks until all work has finished and there are no more handlers to be dispatched, or until the io_service has been interrupted.

Multiple threads may call the run() function to set up a pool of threads from which the io_service may execute handlers. ... "

In addition, an io_service can have a permanent work attached to it which will keep the event processing loop active even if all of the real communication events have been processed.

Does that sound like it would work to accomplish what you suggested? I haven't tried it yet.

I'm not 100% sure, I haven't tried that feature of asio. My guess is that it's a little different in that there may be a relationship between the io requests posted in the thread and where the callbacks are processed....so you have to 'know in advance' how threads and processing map. That's different from a typical thread pool. Anyway, one of us will either need to try it or ask Chris :-) Jeff

Peter Dimov wrote:

Jeff Garland wrote:

...

But you've already fallen into the trap. Just because the computers are next to each other doesn't mean you are going to get an immediate answer. For example, one of the computers you are communicating with might be really busy doing something (swapping perhaps if it's overloaded).

The same can (and does) happen with local calls if the computer is really busy or starts paging.

Right, but the programmer has no way of reasonably detecting and handling that condition from within his program. If it's on a remote machine he can and should because it's a frequent issue with networked programs. Jeff

Peter Dimov wrote:

Jeff Garland wrote:

Right, that was kind of my point. It's not a black and white issue, but

Don't think I said it was...I was simply trying to nudge towards more robust solution paths.

rather a question of frequency or probability. If you can tolerate the slim chances of blocking, you can go with synchronous calls.

No disagreement.

The fact that it's a different machine doesn't increase the chances that much; interprocess calls to the same machine have a similar chance of blocking, except that you now need half the load, all else being equal.

I was using this as one example. The second you go outside of a 'small, all on the same subnet, you control all the clients and servers' examples the odds of these these issues increase alot in my experience.

Async is better but requires more coding investment that may not be worth it, pragmatically speaking.

Right, so my argument is a) beyond some pretty small scale examples sync is doomed, and b) async isn't that much harder, so why not learn how to deal with it up front? When we start talking about RPC facilities for Boost it's just not going to be acceptable to limit the problem space to what sync can do. Jeff

Rene Rivera wrote:

From my own experience (I have my own RMI/RPC framework), I can say that I agree with Peter.

Actually I didn't think Peter and I were disagreeing :-)

I have both sync and async calls and both are beneficial. It is usually a matter of context which one you use. For example I use sync calls during the connection process since in that context nothing else makes much sense.

Which is reasonable assuming 1) you make a connection up front independent of the actual RMI/RPC, 2) you don't have a high latency network (eg: over a satellite hop or two), 3) you control the machines used by both client and server, etc, etc. Of course, you probably don't wait forever for a connection to complete, either, so I assume there's at least a built-in timeout...maybe even a disconnect/reconnect strategy.

Almost everywhere else I use async calls.

Good choice ;-) Really I think there's no disagreement here. I don't have a problem with an RPC option that includes sync behavior. I just don't want Stjepan to go down the path of building and all sync RPC solution b/c I don't think that's going to fly. Jeff

On Mon, 30 Apr 2007 11:48:31 -0700, Stjepan Rajko wrote:

I guess the consensus is that there is use for both sync and async aspects, and both should be

I personally would suggest looking at how futures can make the async/sync elements easier. Imagine I have a function 'rpc' which calls the passed function and arguments remotely and returns a future<T>, where T is the return type. Along the lines of: future<T> rpc(F function, INARG1, ... , INARGN, OUTARG1, ... OUTARGN); 1) Synchronous behavior: cout << "1+2 = " << rpc(add, 1, 2) << "\n"; 2) Async behavior: future<int> fi = rpc(add, 1, 2); ...doing other things... cout << "1+2 = " << fi.get() << "\n"; //will block if fi is not yet complete 3) rpc call fails: future<int> fi = rpc(add, 1, 2); try { cout << "1+2 = " << fi << "\n"; //will throw if fi contains exception } catch (rpc_connect_error &e) { ... } 4) I want to cancel the rpc if it isn't ready yet: future<int> fi = rpc(add, 1, 2); ...doing other things... fi.cancel(); // cancels if not yet complete 5) I want a basic timeout fi.timed_wait(100 /*ms*/); if (!fi.ready()) fi.cancel(); 6) My remote call may return an exception (and my rpc library has some exception transportation mechanism) try { cout << "1+2 = " << fi << "\n"; } catch (...) { cout << "ERROR\n"; } 7) I have both a return value and an "Out" argument: future<int> multiplied; future<int> fi = rpc(add_and_mult, 1, 2, multiplied); cout << "1+2 = " << fi << "\n"; cout << "1*2 = " << multiplied << "\n"; This is the ideal application of futures. My futures library can support all of the above uses today without modification. As a bonus, then any code which uses futures can transparently use the rpc library as a back-end (except for dispatch of course)...a future is a future is a future. I'll work with you as I prepare for boost submission if you want to explore this route. Braddock Gaskill Dockside Vision Inc

On Tue, 01 May 2007 05:19:59 -0700, Sohail Somani wrote:

I second this. I've written a light rpc framework and a futures-like thing was the main point of interaction.

Where is your futures library anyway?

My futures library is at: http://braddock.com/~braddock/future/ It is still early. -braddock

-----Original Message----- From: boost-bounces@lists.boost.org on behalf of Braddock Gaskill Sent: Tue 5/1/2007 4:16 AM To: boost@lists.boost.org Subject: Re: [boost] [rpc/marshal] prototype rpc / marshal libraryusing asio and serialization (and futures in the future?)

On Mon, 30 Apr 2007 11:48:31 -0700, Stjepan Rajko wrote:

...

I guess the consensus is that there is use for both sync and async aspects, and both should be

I personally would suggest looking at how futures can make the async/sync elements easier. _______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost

-----Original Message----- From: boost-bounces@lists.boost.org [mailto:boost-bounces@lists.boost.org] On Behalf Of Stjepan Rajko

Great! Now the rpc framework offers only an asynchronous interface, and the futures are what can make it synchronous... I don't see any drawbacks to this at the moment, and it makes the rpc code a great deal simpler.

One thing I'm not sure of is how you plan to call any function remotely given an *object*. Polymorphic interfaces? I guess any solution would require F to be serializable. Sorry if I missed the documentation (is there some?) Thanks, Sohail

-----Original Message----- From: boost-bounces@lists.boost.org [mailto:boost-bounces@lists.boost.org] On Behalf Of Stjepan Rajko

Yes, each object passed / returned must be serializable via Boost.Serialization. No polymorphic interface is required from the objects.

A call to rpc(f,a1,a2), does it generate some_child<R(T1,T2)> : public some_base which is then serialized and subsequently deserialized and called by some_base *? Thanks, Sohail

Sorry to have left the discussion so early... I'll try to participate a bit more but life has been (and still is) a hell. There are good platforms out there (corba and dcom) that already performs this kind of work. My first goal was more modest than full RPC support, but as I see it you've made a lot of work. I took a look at the code you had two weeks ago, but don't know how it evolved since then. What I'm curious about is this: suppose that you have a class A with a function member M1 (say which takes a ref to an int and change it for, like, system msec since Epoch). If you have a function that coincidentally takes a pointer (or reference) to A and call its member, where is that member called? Both DCOM and CORBA will work with interfaces that are well-defined on both side to circumvent this case and make it work, but what would your code do in that case? I suppose that actually it wouldnt get back to callee but be called on the same computer, which would not be desired behavior. Another case is exception throwing and memory allocation. What if, while un-marshalling parameters on target, you're OutOfMemory? Or worst, on marshalling it back to origin? That latter case may be a lot of problems since some side effects on target may have been done, and should be undone... or not? Will you restrict the case awfully to the point where the user don't have much choice (no exception, const pointer/reference (watch out for those mutable), one out, no in/out, no class passing / member call back, etc)? Yours, Hans On 1-May-07, at 9:25 PM, Stjepan Rajko wrote:

On 5/1/07, Sohail Somani <s.somani@fincad.com> wrote:

...

...

-----Original Message----- From: boost-bounces@lists.boost.org [mailto:boost-bounces@lists.boost.org] On Behalf Of Stjepan Rajko

...

Yes, each object passed / returned must be serializable via Boost.Serialization. No polymorphic interface is required from the objects.

A call to rpc(f,a1,a2), does it generate some_child<R(T1,T2)> : public some_base which is then serialized and subsequently deserialized and called by some_base *?

Sorry, I sent you the wrong link... oops. The actual link is:

dancinghacker.com/code/marshal/

Apologies.

A call rpc(f,a1,a2), to use that notation, is itself modeled roughly as call<R(T1,T2)> this_particular_call(f,a1,a2) which inherits call_base (which does provide a polymorphic interface). f (function identifier), a1 and a2 are all serialized within the call class, and after a a call rpc_client(this_particular_call) and sent over the network. The syntax can be made to look like it's just a local function call, but the guts of it look like what I described.

On the other side, a registry_server receives the packet, unserializes the function id, and calls a marshaled<R(T1,T2)> wrapper of the actual function. The marshaled class takes care of unserializing the parameters and calling the actual function, then the results get marshaled back etc.

Stjepan _______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/ listinfo.cgi/boost

On Tue, 01 May 2007 15:34:47 -0700, Stjepan Rajko wrote:

...

fi.cancel(); // cancels if not yet complete

Cancellation by cancelling the feature is a cool idea... I assume the promise can be notified that all futures have been cancelled via a callback? Speaking of... if a function call has many outstanding promises/futures (e.g., a return value and a few "out" arguments), is there a way to group those all together so that they can be cancelled as a group?

When future<T>::cancel() is called, a future_cancel exception is set in the promise which all future instances are directly or indirectly constructed from. The futures are all placed in a ready() state, with has_exception() true, and will throw future_cancel if get() or implicit conversion is invoked. You can set a simple cancel callback to clean up your side with: promise<T>::set_cancel_handler(const boost::function<void (void)> &f) Now, if you have both a return value future, and a set of OutArg futures, you can handle that in one of two ways: 1) Do nothing. A nifty feature is that when the last promise object goes out of scope before being set, all associated futures are immediately set with a "broken_promise" exception. So when you remove your copy of the promise objects, your waiting callers get a broken_promise. 2) Maintain a list of promises and cancel them all yourself with whatever exception or value you want.

...

5) I want a basic timeout fi.timed_wait(100 /*ms*/); if (!fi.ready()) fi.cancel();

OK, but I think similar functionality should also be added to the rpc interface - it might be useful to be able to set a flat timeout for an individual function (or all functions), and then cancel the promise in

Yeah, I'm sure you'll want a bit more power and probably a few optional parameters in your 'rpc()' call. Maybe: future<T> rpc(RpcParameterSet, F func, ARG1, ..., ARGN) You may also want to not return a future directly, but perhaps return some rpc_call_handle type which is implicitly convertable to a future. rpc_call_handle<T> rpc(...) The rpc_call_handle may have a more powerful rpc-specific interface. Since it is implicitly convertable to a future<T>, the user can either choose to ignore it and accept a simple future<T> with the above usage, or make use of the handle for whatever rpc-specific interface is needed.

I do plan to add exception transportation. Is it possible to have the future throw a custom (specified by the promise) exception? Your point 3) suggests that this is possible also.

Yes, within certain limits. Thank Peter Dimov for his exception_ptr handling. In fact, looking at my code now I think I could improve my implementation to allow throwing arbitrary exception types. This is one of those "early" areas.

...

7) I have both a return value and an "Out" argument: future<int> multiplied; future<int> fi = rpc(add_and_mult, 1, 2, multiplied); cout << "1+2 = " << fi << "\n"; cout << "1*2 = " << multiplied << "\n";

ATM, seems like future doesn't have a default constructor, so future<int> multiplied; doesn't compile.

Heh, sorry about that...you are correct. A future must by constructed from a promise or another future, since a default-constructed future would never get set. Options: 1) change to promise<int> multiplied; and pass the promise to the rpc call. 2) Pass the Out arg future by reference, and replace it with a future constructed from a promise which the rpc() call creates. Work-around for the lack of default constructor would be: future<int> multiplied(promise<int>()); //will immediately be set to broken_promise, but that okay. (or ask me to provide a default constructor which does this, which is probably a good idea) I don't like (1) because it blurs the divide between future holder and promise fulfiller, and breaks the broken_promise mechanism. So I guess I should add that default constructor, eh? :)

Also, if the argument was "InOut" rather than just "Out", we'll need

Well, if you use the pass-by-reference future solution above, then you COULD pass in an already-set future, and then have the rpc() call take that value and then assign the referenced future to a new promise. I'm not sure I like that though, it seems abusive and confusing.

some documentation of the features - I've been getting a lot from the code and the test cases, but sometimes the intended use escapes me.

Yes, I need to write some real documentation. :) Please continue to ask me and I will provide support. I know the implementation will benefit from it. We can move to private e-mail if you prefer (braddock@braddock.com). Braddock Gaskill Dockside Vision Inc

On Sun, 29 Apr 2007 13:56:12 -0700, "Jeff Garland" <jeff@crystalclearsoftware.com> said:

I'm not 100% sure, I haven't tried that feature of asio. My guess is that it's a little different in that there may be a relationship between the io requests posted in the thread and where the callbacks are processed....so you have to 'know in advance' how threads and processing map. That's different from a typical thread pool. Anyway, one of us will either need to try it or ask Chris :-)

Yep, Stjepan's correct. You can have a pool of threads calling io_service::run(), and all those threads are considered equivalent when it comes to invoking the completion handlers. I.e. the io_service will just pick any one of them to run the handler. Cheers, Chris

On Thu, 03 May 2007 16:48:13 -0700, "Jeff Garland" <jeff@crystalclearsoftware.com> said:

Christopher Kohlhoff wrote:

...

Yep, Stjepan's correct. You can have a pool of threads calling io_service::run(), and all those threads are considered equivalent when it comes to invoking the completion handlers. I.e. the io_service will just pick any one of them to run the handler.

Ok, I think the docs might deserve just a bit of clarification on this point. Right now it just says:

Multiple threads may call the run() function to set up a pool of threads from which the io_service may execute handlers.

Fair enough. I'll add my above comments to that bit of the docs.

So are you also saying that io_service guarantees that the once a handler is displatched in a thread another request won't interrupt it?

Not sure I completely follow you here, but the io_service will not nest invocation of handlers unless you explicitly request it (by using io_service::dispatch()).

Also, I think I was a bit thrown off by this part of the docs:

The io_service guarantees that the handler will only be called in a thread in which the run() member function is currently being invoked. The handler may be executed inside this function if the guarantee can be met.

which now I'm really not sure I understand :-/

From memory, this text is under io_service::dispatch(), and "this function" refers to io_service::dispatch(). It means that io_service::dispatch works like this:

- Am I in a thread in which io_service::run() is being called? - If yes, invoke handler immediately. - If no, use io_service::post() to defer invocation of handler.

One of the 'interesting' side effects of this behavior, btw, is that you better not write any code where you do something like this:

thread 1 -> a) cretate io_service, b) setup handlers, c) call run; thread 2 -> a) setup handlers, b) do other initialization c) call run;

In thread 2 the handlers will be 'immediately active' because thread 1 has called run already...so the callbacks might happen before the required stop b initialization -- it really needs to be done first.

Yep. In more complicated thread-pool-based use cases I would probably use an io_service::strand to address this. I.e. from thread 2: my_strand.post(setup_stuff); my_io_service.run(); void setup_stuff() { setup handlers to execute in my_strand do other initialisation } This will ensure that the I/O completion handlers aren't called until after setup_stuff() finishes. Cheers, Chris