[bind] & [function] How do I store member functions with generalised arguments for later invocation?
I am wanting to write library code that will allow a user to create an "event" which is associated with a thread, a class instance, and a member function of that class. The user can then "post" data (generalised number of arguments) - which results in the class instance's member function being called in the context of the associated thread. For example (pseudo code): event { thread* thr; class &that; mem_fun* fun; void post(); void post(a1); void post(a1, a2); void post(a1, a2, a3, ...etc); } mem_fun* fun stores a member function pointer which is of the form void class::fun(...0, 1 or more arguments...) Here is some example code which uses boost::bind to bind arguments to member functions and posts them to be called on another thread. However, it does not go far enough. #include <boost/function.hpp>
#include <boost/bind.hpp> #include <boost/enable_shared_from_this.hpp> #include <boost/thread/thread.hpp> #include <boost/thread/mutex.hpp> #include <boost/thread/condition.hpp> #include <list> #include <stdio.h>
// cross-thread event queue. multiple threads can post jobs, one or many threads can execute // jobs. class EventQueue { public: typedef boost::function<void()> Job;
// puts a jon into the queue void post(Job job) { boost::mutex::scoped_lock lock(mtx_); jobs_.push_back(job); cnd_.notify_one(); }
// pulls one job from the queue, returns false when stopped bool pull(Job* job) { boost::mutex::scoped_lock lock(mtx_); for(;;) { // handle spurious wake-ups while(!stop_ && jobs_.empty()) cnd_.wait(lock); if(!jobs_.empty() && 2 != stop_) { job->swap(jobs_.front()); // move efficiently, avoiding *job = jobs.front() jobs_.pop_front(); return true; } else if(stop_) { return false; } } }
// make pull() return false void stop(bool cancel_jobs) { boost::mutex::scoped_lock lock(mtx_); stop_ = 1 + cancel_jobs; // 1 - complete jobs, 2 - cancel jobs cnd_.notify_all(); }
EventQueue() : stop_() {} ~EventQueue() { this->stop(true); }
private: boost::mutex mtx_; boost::condition cnd_; typedef std::list<Job> Jobs; Jobs jobs_; int stop_; };
struct JobX // micro-oprimization, embed the reference counter into the job to avoid an extra memory // allocation by boost::shared_ptr ctor : boost::enable_shared_from_this<JobX> { void foo() { printf("%p foo()\n", this); }
void bar(int a) { printf("%p bar(%d)\n", this, a); } };
void anotherThread(EventQueue* queue) { EventQueue::Job job; // wait and execute jobs till stopped while(queue->pull(&job)) job(); // execute the job }
int main() { EventQueue queue; // start another thread and pass an argument boost::thread another_thread(boost::bind(anotherThread, &queue));
// post jobs, allocate in this thread, deallocate in the other boost::shared_ptr<JobX> job_0(new JobX); queue.post(boost::bind(&JobX::foo, job_0)); // post several jobs to the same object, deallocated when no longer in use boost::shared_ptr<JobX> job_1(new JobX); queue.post(boost::bind(&JobX::foo, job_1)); queue.post(boost::bind(&JobX::bar, job_1, 1)); // pass an extra arg 1 queue.post(boost::bind(&JobX::bar, job_1, 2)); // pass an extra arg 2
// stop the queue and let it complete all jobs queue.stop(false); another_thread.join(); }
And this is where I fall down. I am able to bind arguments to JobX::bar above, at the point at which I create my boost::function<void()>, but if I want to bind _1, _2 etc arguments for calling later, I'm not sure what to do. So my question is, how should I form class Event such that I can store member functions of the form void T::fn(), void T::fn(a1), void T::fn(a1, a2, etc...), and then allow a generalised form of post(), post(a1), post(a1, a2, etc...) to bind the arguments given in post to the callback and post it to the queue? I have some code below which clearly won't compile, but should illustrate to some degree what I'm looking for. template <typename T> class Event { EventQueue &queue_; // the event queue which will process the job boost::shared_ptr<T> that_; // the object whose mem_fun the event queue will call typedef boost::function<void (T*, void) > CB_0 ; // used for callbacks which are of the form void T::fn(); typedef boost::function<void (T*, _1) > CB_1 ; // used for callbacks which are of the form void T::fn(1 arg); typedef boost::function<void (T*, _1, _2) > CB_2 ; // used for callbacks which are of the form void T::fn(1 arg); CB_0 cb_0; CB_1 cb_1; CB_2 cb_2; public: Event(boost::shared_ptr<T> &that, CB &cb, EventQueue &queue) : that_(that), cb_(cb), queue_(queue) {} void post() { queue_.post(boost::bind(&cb_0, that)); template <typename arg_1> void post(arg_1 &arg1) { queue_.post(boost::bind(&cb_1, that)); template <typename arg_1, typename arg_2> void post(arg_1 &arg1, arg_2 &arg2) { queue_.post(boost::bind(&cb_2, that)); // TODO: args should be shared_ptr }; TIA Steve
Steve Lorimer wrote:
I am wanting to write library code that will allow a user to create an "event" which is associated with a thread, a class instance, and a member function of that class. The user can then "post" data (generalised number of arguments) - which results in the class instance's member function being called in the context of the associated thread.
Steve, sorry if this is a stupid question, but by what delta does Boost.Signals2 fall short of your requirements?
Thanks Nat, I didn't know about this yet (noob!)
From a quick perusal (almost bed time :) !) I can't see how I would achieve my aim, which is:
1. create the "event" (signal) which is associated with a member function "callback" (slot). Associate a thread with the event (can I create a signal connected to a slot which is a member function?) 2. in any other thread, call "post" (sig(data)) where data is a shared_ptr 3. the callback (slot) is called, running in the context of the *other* thread (the one originally associated with the "event" / (slot). I'm trying to encapsulate a generic inter-thread messaging system where objects are passed around between threads, and associated with a member function which is designed to process said objects. This design is essentially a messaging system between threads - for a simple example - we have a producer/consumer thread model. The consumer thread is encapsulated in a class (eg: class consumer_thread), which has a member function (eg: void consume(struct data)). The producer creates an "event" object which is associated with the consumer_thread object, it's void consume(struct data) member function, and the thread which must process the data (consumer_thread). Now when the producer produces some data, it calls event->post(data), which queues up an object which records the class instance and it's member function which must be called with parameter "data", and a shared_ptr to data. The consumer thread wakes up, gets the top of the queue, and calls obj->mem_fun(data) --> ergo, mem_fun is called in the correct thread context, and has the argument "data" to consume. Does that make any sense? I don't think I'm explaining it well. Any suggestions, (how to do what I'm asking for, or trashing my design and offering a better design), would be greatly appreciated! TIA Steve On 2 June 2010 19:06, Nat Goodspeed <nat@lindenlab.com> wrote:
Steve Lorimer wrote:
I am wanting to write library code that will allow a user to create an
"event" which is associated with a thread, a class instance, and a member function of that class. The user can then "post" data (generalised number of arguments) - which results in the class instance's member function being called in the context of the associated thread.
Steve, sorry if this is a stupid question, but by what delta does Boost.Signals2 fall short of your requirements?
_______________________________________________ Boost-users mailing list Boost-users@lists.boost.org http://lists.boost.org/mailman/listinfo.cgi/boost-users
I'm thinking of the same thing, and would reach for std::tr1::function generalized function objects to be the callback object queued between threads. between 'function' and 'bind', both standard now, it should do the trick. This will play nicely with compiler-native lambdas when they are available to your project, too. From: boost-users-bounces@lists.boost.org [mailto:boost-users-bounces@lists.boost.org] On Behalf Of Steve Lorimer Sent: Wednesday, June 02, 2010 4:36 PM To: boost-users@lists.boost.org Subject: Re: [Boost-users] [bind] & [function] How do I store member functions with generalised arguments for later invocation? Thanks Nat, I didn't know about this yet (noob!)
From a quick perusal (almost bed time :) !) I can't see how I would achieve my aim, which is:
1. create the "event" (signal) which is associated with a member function "callback" (slot). Associate a thread with the event (can I create a signal connected to a slot which is a member function?) 2. in any other thread, call "post" (sig(data)) where data is a shared_ptr 3. the callback (slot) is called, running in the context of the *other* thread (the one originally associated with the "event" / (slot). I'm trying to encapsulate a generic inter-thread messaging system where objects are passed around between threads, and associated with a member function which is designed to process said objects. This design is essentially a messaging system between threads - for a simple example - we have a producer/consumer thread model. The consumer thread is encapsulated in a class (eg: class consumer_thread), which has a member function (eg: void consume(struct data)). The producer creates an "event" object which is associated with the consumer_thread object, it's void consume(struct data) member function, and the thread which must process the data (consumer_thread). Now when the producer produces some data, it calls event->post(data), which queues up an object which records the class instance and it's member function which must be called with parameter "data", and a shared_ptr to data. The consumer thread wakes up, gets the top of the queue, and calls obj->mem_fun(data) --> ergo, mem_fun is called in the correct thread context, and has the argument "data" to consume. Does that make any sense? I don't think I'm explaining it well. Any suggestions, (how to do what I'm asking for, or trashing my design and offering a better design), would be greatly appreciated! TIA Steve On 2 June 2010 19:06, Nat Goodspeed <nat@lindenlab.com<mailto:nat@lindenlab.com>> wrote: Steve Lorimer wrote: I am wanting to write library code that will allow a user to create an "event" which is associated with a thread, a class instance, and a member function of that class. The user can then "post" data (generalised number of arguments) - which results in the class instance's member function being called in the context of the associated thread. Steve, sorry if this is a stupid question, but by what delta does Boost.Signals2 fall short of your requirements? _______________________________________________ Boost-users mailing list Boost-users@lists.boost.org<mailto:Boost-users@lists.boost.org> http://lists.boost.org/mailman/listinfo.cgi/boost-users TradeStation Group, Inc. is a publicly-traded holding company (NASDAQ GS: TRAD) of three operating subsidiaries, TradeStation Securities, Inc. (Member NYSE, FINRA, SIPC and NFA), TradeStation Technologies, Inc., a trading software and subscription company, and TradeStation Europe Limited, a United Kingdom, FSA-authorized introducing brokerage firm. None of these companies provides trading or investment advice, recommendations or endorsements of any kind. The information transmitted is intended only for the person or entity to which it is addressed and may contain confidential and/or privileged material. Any review, retransmission, dissemination or other use of, or taking of any action in reliance upon, this information by persons or entities other than the intended recipient is prohibited. If you received this in error, please contact the sender and delete the material from any computer.
*From:* boost-users-bounces@lists.boost.org [mailto:boost-users-bounces@lists.boost.org] *On Behalf Of *Steve Lorimer *Sent:* Wednesday, June 02, 2010 4:36 PM *To:* boost-users@lists.boost.org *Subject:* Re: [Boost-users] [bind] & [function] How do I store member functions with generalised arguments for later invocation?
From a quick perusal (almost bed time :) !) I can't see how I would achieve my aim, which is:
1. create the "event" (signal) which is associated with a member function "callback" (slot). Associate a thread with the event
(can I create a signal connected to a slot which is a member function?)
Yes, you can either use boost::bind() explicitly (or the Boost Lambda Library or Boost Phoenix), or I believe boost::signals2::signal::connect() will accept bind-like arguments and call boost::bind() internally.
2. in any other thread, call "post" (sig(data)) where data is a shared_ptr
3. the callback (slot) is called, running in the context of the *other* thread (the one originally associated with the "event" / (slot).
I'm trying to encapsulate a generic inter-thread messaging system where objects are passed around between threads, and associated with a member function which is designed to process said objects.
The producer creates an "event" object which is associated with the consumer_thread object, it's void consume(struct data) member function, and the thread which must process the data (consumer_thread). Now when the producer produces some data, it calls event->post(data), which queues up an object which records the class instance and it's member function which must be called with parameter "data", and a shared_ptr to data. The consumer thread wakes up, gets the top of the queue, and calls obj->mem_fun(data) --> ergo, mem_fun is called in the correct thread context, and has the argument "data" to consume.
John Dlugosz wrote:
I’m thinking of the same thing, and would reach for std::tr1::function generalized function objects to be the callback object queued between threads.
I don't quite understand your requirements, but let me take a shot. The important point that John brings out is that if you want the consumer to run code on its own thread, rather than running on the producer's thread, then I think we're talking about a queue. This would be instead of using Boost.Signals2 at all, which would call its slots immediately on the producer's thread. So you instantiate a queue and start the consumer thread waiting for items to arrive on that queue. I agree with John that it sounds like the "items" in question are std::tr1::function objects (or I'd say boost::function objects, but whatever). When you say:
The consumer thread wakes up, gets the top of the queue, and calls obj->mem_fun(data)
it's not really clear to me whether 'obj' is owned by the consumer thread or is in the hands of the producer object. Let's consider both cases. ==== Either way: class Data { ... }; typedef boost::shared_ptr<Data> DataPtr; ==== If 'obj' is the ConsumerClass instance: class ConsumerClass { public: ... void run(); void doSomething(DataPtr data); void somethingElse(DataPtr data); ... }; typedef (ConsumerClass::*ConsumerMethod)(DataPtr); typedef boost::function<void(ConsumerClass*)> QueueItem; typedef YourFavoriteThreadSafeQueue<QueueItem> Queue; void ConsumerClass::run() { for (;;) { QueueItem item(queue.get()); item(this); } } void ProducerClass::post(ConsumerMethod method, DataPtr data) { queue.put(boost::bind(method, _1, data)); } ==== If 'obj' is owned instead by the producer thread: class ConsumerClass { public: ... void run(); ... }; class ProcessorClass { public: ... void doSomething(DataPtr data); void somethingElse(DataPtr data); ... }; typedef (ProcessorClass::*ProcessorMethod)(DataPtr); typedef boost::function<void()> QueueItem; typedef YourFavoriteThreadSafeQueue<QueueItem> Queue; void ConsumerClass::run() { for (;;) { QueueItem item(queue.get()); item(); } } void ProducerClass::post(ProcessorClass& obj, ProcessorMethod method, DataPtr data) { queue.put(boost::bind(method, obj, data)); } ==== If you have a number of consumer threads, and you want the producer to select among them, I'd associate a separate queue with each thread and have the producer put items to the appropriate queue, in effect implying the consumer thread by the producer's choice of queue. I'd better stop here and let you explain what parts I've badly misunderstood.
Hi Nat I think we're on the same page here - but I'm looking for a more generalised form. Who owns "obj" or T that_ in my example below is unimportant - I guess we should enforce using a shared_ptr to "obj" so that it is unimportant who owns it. So this is what I came up with before your reply... and although I think it's not really the right way to do it, I think it will serve to illustrate what I'm trying to do. (this code works in conjunction with the EventQueue code I pasted in my original question (see below) // all callbacks will be of the form void T::mem_fn(...) // this for the callback of form T::mem_fn() template <typename T> class event_0 { EventQueue &queue_; // the event queue which will process the job boost::shared_ptr<T> that_; // the object whose member function the event queue will call typedef void (T::*CB_t)(); // the member function of class T which we will callback to boost::function<void()> job_; public: event_0(boost::shared_ptr<T> that, CB_t cb, EventQueue &queue) : queue_(queue), that_(that), job_(boost::bind(cb, that)) // bind the class instance and its callback function together for later invocation {} // the function the user calls which posts the job to the queue. The thread associated with the queue picks up the // job off the queue and calls it (ergo - calls that_->CB_t(); ) void post() { queue_.post(job_); } }; // So in my application I have this class: struct foo0 : boost::enable_shared_from_this<foo0> { event_0<foo0> ev; foo0(EventQueue &queue) : ev(boost::shared_ptr<foo0>(this), &foo0::bar, queue) { } void bar() { printf("%p foo::bar()\n", this); } ~foo0() { std::cout << __func__ << " - Thread id is: " << boost::this_thread::get_id() << std::endl; } }; *// and I create an event which is associated with a queue. This is most likely a class member, where the class is, for eg, the producer, and the queue being passed to the event, the queue for the consumer.* boost::shared_ptr<foo0> job_3(new foo0(queue)); *// and then some time later, when the producer has created some data...* job_3->ev.post(); // resulting in void foo0::bar() function being called on the consumer thread *// so - now we want to be able to post a shared_ptr to the consumer - ie: the producer creates some data (maybe receives it from a socket) and wants to pass it to the consumer. So we want to be able to go ** job_4->ev.post(data);* // this for the callback of form T::mem_fn(A0) template <typename T, typename A0> class event_1 { EventQueue &queue_; // the event queue which will process the job boost::shared_ptr<T> that_; // the object whose mem_fun the event queue will call typedef void (T::*CB_t)(A0); typedef boost::function<void()> Job; Job job_; public: event_1(boost::shared_ptr<T> that, CB_t cb, EventQueue &queue) : queue_(queue), that_(that), job_(boost::bind(cb, that, _1)) {} void post(A0 data) { queue_.post(boost::bind(job_, data)); } }; In other words, I hide the details of the bind from the user, and all he has to do is create some data and post it to the event, which sorts out the binding, queueing, etc - We could make it requisite that all classes which have a callback function used by the event must be derived from shared_ptr (or be shared_ptrs, or whatever), and all data posted by events ( event_1::post(A0) ) must be shared - to ensure thread safety. Ideally I'd like to be able to generalise the above somehow, and be able to cater for post(A0), post(A0, A1), post(A0, A1, A2) etc. Do you sorta see what I'm trying to do? If I can attempt to explain it better... I'm trying to create some library code which will handle inter-thread messaging in an elegant way, and hide the implementation details from the user. The idea is the EventQueue class will actually be part of an EventThread class - you create an EventThread, and all it does is endlessly loop, picking up Event objects from its EventQueue, and calling their functors (which in fact wrap up a class and member callback function and has a shared_ptr to some data which is the argument to the member callback function) - So the creation of the thread, the creation of the queue etc is hidden - all the user does is create events and call event::post(data). Do you think this is even a good design? Would you suggest a different design philosophy? TIA Steve PS: My EventQueue code is here: #include <boost/enable_shared_from_this.hpp> #include <boost/thread/thread.hpp> #include <boost/thread/mutex.hpp> #include <boost/thread/condition.hpp> #include <boost/function.hpp> #include <boost/bind.hpp> #include <stdio.h> #include <list> #include <iostream> // cross-thread event queue. multiple threads can post jobs, one or many threads can execute // jobs. class EventQueue { public: typedef boost::function<void()> Job; private: boost::mutex mtx_; boost::condition cnd_; typedef std::list<Job> Jobs; Jobs jobs_; int stop_; public: // puts a jon into the queue void post(Job job) { boost::mutex::scoped_lock lock(mtx_); jobs_.push_back(job); cnd_.notify_one(); } // pulls one job from the queue, returns false when stopped bool pull(Job* job) { boost::mutex::scoped_lock lock(mtx_); for(;;) { // handle spurious wake-ups while(!stop_ && jobs_.empty()) cnd_.wait(lock); if(!jobs_.empty() && 2 != stop_) { job->swap(jobs_.front()); // move efficiently, avoiding *job = jobs.front() jobs_.pop_front(); return true; } else if(stop_) { return false; } } } // make pull() return false void stop(bool cancel_jobs) { boost::mutex::scoped_lock lock(mtx_); stop_ = 1 + cancel_jobs; // 1 - complete jobs, 2 - cancel jobs cnd_.notify_all(); } EventQueue() : stop_() {} ~EventQueue() { this->stop(true); } }; //----------------------------------------------------------------------- // all callbacks will be of the form void T::mem_fn(...) // this for the callback of form T::mem_fn() template <typename T> class event_0 { EventQueue &queue_; // the event queue which will process the job boost::shared_ptr<T> that_; // the object whose mem_fun the event queue will call typedef void (T::*CB_t)(); typedef boost::function<void()> Job; Job job_; public: event_0(boost::shared_ptr<T> that, CB_t cb, EventQueue &queue) : queue_(queue), that_(that), job_(boost::bind(cb, that)) {} void post() { queue_.post(job_); } }; //----------------------------------------------------------------------- struct foo0 : boost::enable_shared_from_this<foo0> { event_0<foo0> ev; foo0(EventQueue &queue) : ev(boost::shared_ptr<foo0>(this), &foo0::bar, queue) { } void bar() { printf("%p foo::bar()\n", this); } ~foo0() { std::cout << __func__ << " - Thread id is: " << boost::this_thread::get_id() << std::endl; } }; //----------------------------------------------------------------------- void anotherThread(EventQueue* queue) { std::cout << __func__ << " - Thread id is: " << boost::this_thread::get_id() << std::endl; EventQueue::Job job; // wait and execute jobs till stopped while(queue->pull(&job)) job(); // execute the job } //----------------------------------------------------------------------- int main() { std::cout << __func__ << " - Thread id is: " << boost::this_thread::get_id() << std::endl; EventQueue queue; // start another thread and pass an argument boost::thread another_thread(boost::bind(anotherThread, &queue)); boost::shared_ptr<foo0> job_3(new foo0(queue)); job_3->ev.post(); // stop the queue and let it complete all jobs queue.stop(false); another_thread.join(); return 0; } //----------------------------------------------------------------------- On 3 June 2010 22:11, Nat Goodspeed <nat@lindenlab.com> wrote: it's not really clear to me whether 'obj' is owned by the consumer thread or
is in the hands of the producer object. Let's consider both cases.
==== Either way:
class Data { ... }; typedef boost::shared_ptr<Data> DataPtr;
==== If 'obj' is the ConsumerClass instance:
class ConsumerClass { public: ... void run(); void doSomething(DataPtr data); void somethingElse(DataPtr data); ... };
typedef (ConsumerClass::*ConsumerMethod)(DataPtr);
typedef boost::function<void(ConsumerClass*)> QueueItem; typedef YourFavoriteThreadSafeQueue<QueueItem> Queue;
void ConsumerClass::run() { for (;;) { QueueItem item(queue.get()); item(this); } }
void ProducerClass::post(ConsumerMethod method, DataPtr data) { queue.put(boost::bind(method, _1, data)); }
Expanding on my previous post, I thought a way to create a generalised version of my "event" class, would be to have a single "event" class which has a boost::function to store a functor created using boost::bind on my object and member funciton. I'd then use templatised helper functions for the general forms of using boost::bind. See below: // typedef generalised callback types template<class T> struct cb0 { typedef void (T::*type)(); // define a type for member functions of the form 'void T::fn()' }; template<class T, typename A0> struct cb1 { typedef void (T::*type)(boost::shared_ptr<A0>); // define a type for member functions of the form 'void T::fn(shared_ptr<A0>)' }; //----------------------------------------------------------------------- // the event class which keeps track of our job and the queue we want to post the job to // all private so we can only use our friend helper functions class event { EventQueue &queue_; // the event queue which will process the job typedef boost::function<void()> Job; // the generalised job we will post to the queue Job job_; event(EventQueue &queue, Job &job) : queue_(queue), job_(job) { } // event creation friends template<typename T> friend boost::shared_ptr<event> create_event(boost::shared_ptr<T> that, cb0<T>::type cb, EventQueue &queue); template<typename T, typename A0> friend boost::shared_ptr<event> create_event(boost::shared_ptr<T> that, cb1<T, A0>::type cb, EventQueue &queue); // event post friends friend void post_to_event(boost::shared_ptr<event> &ev); template<typename A0> friend void post_to_event(boost::shared_ptr<event> &ev, boost::shared_ptr<A0> &data); }; //----------------------------------------------------------------------- // helper functions which create new events template<typename T> boost::shared_ptr<event> create_event(boost::shared_ptr<T> that, cb0<T>::type cb, EventQueue &queue) { // encapsulate an object and it's member function using bind return boost::shared_ptr<event>(new event(queue, boost::bind(cb, that))); } template<typename T, typename A0> boost::shared_ptr<event> create_event(boost::shared_ptr<T> that, cb1<T, A0>::type cb, EventQueue &queue) { // encapsulate an object and it's member function using bind, placeholder for argument to come return boost::shared_ptr<event>(new event(queue, boost::bind(cb, that, _1))); } //----------------------------------------------------------------------- // helper functions which post events onto the event's queue void post_to_event(boost::shared_ptr<event> &ev) { ev->queue_.post(ev->job_); } template<typename A0> void post_to_event(boost::shared_ptr<event> &ev, boost::shared_ptr<A0> &data) { // bind the data to the job ev->queue_.post(boost::bind(ev->job_, data)); } //----------------------------------------------------------------------- However, this doesn't compile, g++ comes back with 'cb0::type' is not a type and 'cb1::type' is not a type... any ideas on how I could fix this? TIA Steve On 4 June 2010 09:33, Steve Lorimer <steve.lorimer@gmail.com> wrote:
Hi Nat
I think we're on the same page here - but I'm looking for a more generalised form. Who owns "obj" or T that_ in my example below is unimportant - I guess we should enforce using a shared_ptr to "obj" so that it is unimportant who owns it.
So this is what I came up with before your reply... and although I think it's not really the right way to do it, I think it will serve to illustrate what I'm trying to do. (this code works in conjunction with the EventQueue code I pasted in my original question (see below)
// all callbacks will be of the form void T::mem_fn(...) // this for the callback of form T::mem_fn() template <typename T> class event_0 { EventQueue &queue_; // the event queue which will process the job boost::shared_ptr<T> that_; // the object whose member function the event queue will call
typedef void (T::*CB_t)(); // the member function of class T which we will callback to
boost::function<void()> job_;
public: event_0(boost::shared_ptr<T> that, CB_t cb, EventQueue &queue) : queue_(queue), that_(that), job_(boost::bind(cb, that)) // bind the class instance and its callback function together for later invocation {}
// the function the user calls which posts the job to the queue. The thread associated with the queue picks up the // job off the queue and calls it (ergo - calls that_->CB_t(); ) void post() { queue_.post(job_); } };
// So in my application I have this class:
struct foo0 : boost::enable_shared_from_this<foo0> { event_0<foo0> ev;
foo0(EventQueue &queue) : ev(boost::shared_ptr<foo0>(this), &foo0::bar, queue) { }
void bar() { printf("%p foo::bar()\n", this); }
~foo0() { std::cout << __func__ << " - Thread id is: " << boost::this_thread::get_id() << std::endl; }
};
*// and I create an event which is associated with a queue. This is most likely a class member, where the class is, for eg, the producer, and the queue being passed to the event, the queue for the consumer.*
boost::shared_ptr<foo0> job_3(new foo0(queue));
*// and then some time later, when the producer has created some data...*
job_3->ev.post(); // resulting in void foo0::bar() function being called on the consumer thread
*// so - now we want to be able to post a shared_ptr to the consumer - ie: the producer creates some data (maybe receives it from a socket) and wants to pass it to the consumer. So we want to be able to go ** job_4->ev.post(data);*
// this for the callback of form T::mem_fn(A0) template <typename T, typename A0> class event_1 { EventQueue &queue_; // the event queue which will process the job boost::shared_ptr<T> that_; // the object whose mem_fun the event queue will call
typedef void (T::*CB_t)(A0); typedef boost::function<void()> Job; Job job_;
public: event_1(boost::shared_ptr<T> that, CB_t cb, EventQueue &queue) : queue_(queue), that_(that), job_(boost::bind(cb, that, _1)) {}
void post(A0 data) { queue_.post(boost::bind(job_, data)); } };
In other words, I hide the details of the bind from the user, and all he has to do is create some data and post it to the event, which sorts out the binding, queueing, etc - We could make it requisite that all classes which have a callback function used by the event must be derived from shared_ptr (or be shared_ptrs, or whatever), and all data posted by events ( event_1::post(A0) ) must be shared - to ensure thread safety.
Ideally I'd like to be able to generalise the above somehow, and be able to cater for post(A0), post(A0, A1), post(A0, A1, A2) etc.
Do you sorta see what I'm trying to do? If I can attempt to explain it better... I'm trying to create some library code which will handle inter-thread messaging in an elegant way, and hide the implementation details from the user. The idea is the EventQueue class will actually be part of an EventThread class - you create an EventThread, and all it does is endlessly loop, picking up Event objects from its EventQueue, and calling their functors (which in fact wrap up a class and member callback function and has a shared_ptr to some data which is the argument to the member callback function) - So the creation of the thread, the creation of the queue etc is hidden - all the user does is create events and call event::post(data).
Do you think this is even a good design? Would you suggest a different design philosophy?
TIA Steve
PS: My EventQueue code is here:
#include <boost/enable_shared_from_this.hpp> #include <boost/thread/thread.hpp> #include <boost/thread/mutex.hpp> #include <boost/thread/condition.hpp> #include <boost/function.hpp> #include <boost/bind.hpp> #include <stdio.h> #include <list> #include <iostream>
// cross-thread event queue. multiple threads can post jobs, one or many threads can execute // jobs. class EventQueue { public: typedef boost::function<void()> Job;
private:
boost::mutex mtx_; boost::condition cnd_; typedef std::list<Job> Jobs; Jobs jobs_; int stop_;
public:
// puts a jon into the queue void post(Job job) { boost::mutex::scoped_lock lock(mtx_); jobs_.push_back(job); cnd_.notify_one(); }
// pulls one job from the queue, returns false when stopped bool pull(Job* job) { boost::mutex::scoped_lock lock(mtx_); for(;;) { // handle spurious wake-ups while(!stop_ && jobs_.empty()) cnd_.wait(lock); if(!jobs_.empty() && 2 != stop_) { job->swap(jobs_.front()); // move efficiently, avoiding *job = jobs.front() jobs_.pop_front(); return true; } else if(stop_) { return false; } } }
// make pull() return false void stop(bool cancel_jobs) { boost::mutex::scoped_lock lock(mtx_); stop_ = 1 + cancel_jobs; // 1 - complete jobs, 2 - cancel jobs cnd_.notify_all(); }
EventQueue() : stop_() {} ~EventQueue() { this->stop(true); } }; //-----------------------------------------------------------------------
// all callbacks will be of the form void T::mem_fn(...) // this for the callback of form T::mem_fn() template <typename T> class event_0 { EventQueue &queue_; // the event queue which will process the job boost::shared_ptr<T> that_; // the object whose mem_fun the event queue will call
typedef void (T::*CB_t)(); typedef boost::function<void()> Job; Job job_;
public: event_0(boost::shared_ptr<T> that, CB_t cb, EventQueue &queue) : queue_(queue), that_(that), job_(boost::bind(cb, that)) {}
void post() { queue_.post(job_); } }; //-----------------------------------------------------------------------
struct foo0 : boost::enable_shared_from_this<foo0> { event_0<foo0> ev;
foo0(EventQueue &queue) : ev(boost::shared_ptr<foo0>(this), &foo0::bar, queue) { }
void bar() { printf("%p foo::bar()\n", this); }
~foo0() { std::cout << __func__ << " - Thread id is: " << boost::this_thread::get_id() << std::endl; }
}; //-----------------------------------------------------------------------
void anotherThread(EventQueue* queue) { std::cout << __func__ << " - Thread id is: " << boost::this_thread::get_id() << std::endl; EventQueue::Job job; // wait and execute jobs till stopped while(queue->pull(&job)) job(); // execute the job } //-----------------------------------------------------------------------
int main() { std::cout << __func__ << " - Thread id is: " << boost::this_thread::get_id() << std::endl;
EventQueue queue; // start another thread and pass an argument boost::thread another_thread(boost::bind(anotherThread, &queue));
boost::shared_ptr<foo0> job_3(new foo0(queue)); job_3->ev.post();
// stop the queue and let it complete all jobs queue.stop(false); another_thread.join();
return 0; } //-----------------------------------------------------------------------
On 3 June 2010 22:11, Nat Goodspeed <nat@lindenlab.com> wrote:
it's not really clear to me whether 'obj' is owned by the consumer thread
or is in the hands of the producer object. Let's consider both cases.
==== Either way:
class Data { ... }; typedef boost::shared_ptr<Data> DataPtr;
==== If 'obj' is the ConsumerClass instance:
class ConsumerClass { public: ... void run(); void doSomething(DataPtr data); void somethingElse(DataPtr data); ... };
typedef (ConsumerClass::*ConsumerMethod)(DataPtr);
typedef boost::function<void(ConsumerClass*)> QueueItem; typedef YourFavoriteThreadSafeQueue<QueueItem> Queue;
void ConsumerClass::run() { for (;;) { QueueItem item(queue.get()); item(this); } }
void ProducerClass::post(ConsumerMethod method, DataPtr data) { queue.put(boost::bind(method, _1, data)); }
Steve Lorimer wrote:
a way to create a generalised version of my "event" class, would be to have a single "event" class which has a boost::function to store a functor created using boost::bind on my object and member funciton.
This is what I gleaned from your previous post, yes: the idea that you can define a single boost::function signature for your queue items because all the binding really happens on the producer side.
I'd then use templatised helper functions for the general forms of using boost::bind.
Well, let me ask you this question. How important is it to you to hide use of boost::bind (or std::tr1::bind?) from your coders? My organization had a big discussion about this a couple years ago. Those who came to C++ from Java, and had no experience with (e.g.) Python, found function/bind upsetting and magical and wanted it safely buried beneath an API layer of our own. Those who have grown accustomed to the power of the Callable concept are used to boost::bind() expressions. We ultimately decided that reading boost::bind() expressions was less ugly than the code it would take to conceal them. The call to our API would resemble the boost::bind() call anyway. That said, boost::signals2::slot (in boost/signals2/slot.hpp) does wrap a bind() call in a way that you might adapt if this is what you want.
Nat Goodspeed wrote:
boost::signals2::slot (in boost/signals2/slot.hpp) does wrap a bind() call in a way that you might adapt if this is what you want.
I intended to add: if you don't mind requiring a C++1x-compliant compiler, you could use perfect forwarding instead.
Oooh, what's perfect forwarding? Anyway, back to your previous question - no, I don't *hav*e to hide the boost::bind etc, but I think my concept will wrap up a lot of functionality rather nicely, and I don't think the calling point will look too ugly. This is what I expect the creation of an event object to look like (using the code detailed below) shared_ptr<event> ev = create_event<foo, bar>(obj, &foo::func, queue); // this for class *foo*, which has member *void foo::func(bar)* which we want to callback to To post an event to be processed by another queue will look like this post_to_event<bar>(ev, data); // posts an object *data* of type *foo* to event *ev* So here's the code that I have come up with - doesn't compile though (of course!) - what do you think? // cross-thread event queue. multiple threads can post jobs, one or many threads can execute// jobs.class EventQueue {public: typedef boost::function<void()> Job; private: boost::mutex mtx_; boost::condition cnd_; typedef std::list<Job> Jobs; Jobs jobs_; int stop_; // puts a job into the queue void post(Job job) { boost::mutex::scoped_lock lock(mtx_); jobs_.push_back(job); cnd_.notify_one(); } public: // pulls one job from the queue, returns false when stopped bool pull(Job* job) { boost::mutex::scoped_lock lock(mtx_); for(;;) { // handle spurious wake-ups while(!stop_ && jobs_.empty()) cnd_.wait(lock); if(!jobs_.empty() && 2 != stop_) { job->swap(jobs_.front()); // move efficiently, avoiding *job = jobs.front() jobs_.pop_front(); return true; } else if(stop_) { return false; } } } // make pull() return false void stop(bool cancel_jobs) { boost::mutex::scoped_lock lock(mtx_); stop_ = 1 + cancel_jobs; // 1 - complete jobs, 2 - cancel jobs cnd_.notify_all(); } EventQueue() : stop_() {} ~EventQueue() { this->stop(true); } // event post friends friend void post_to_event(boost::shared_ptr<event> &ev); template<typename A0> friend void post_to_event(boost::shared_ptr<event> &ev, boost::shared_ptr<A0> &data); };//----------------------------------------------------------------------- // typedef generalised callback typestemplate<class T>struct cb0 { typedef void (T::*type)(); // member function of the form 'T::fn()' }; template<class T, typename A0>struct cb1 { typedef void (T::*type)(boost::shared_ptr<A0>); // member function of the form 'T::fn(shared_ptr<A0>)' };//----------------------------------------------------------------------- // the event class which keeps track of our job and the queue we want to post the job toclass event { EventQueue &queue_; // the event queue which will process the job EventQueue::Job job_; event(EventQueue &queue, EventQueue::Job &job) : queue_(queue), job_(job) { } // event creation friends template<typename T> friend boost::shared_ptr<event> create_event(boost::shared_ptr<T> that, typename cb0<T>::type cb, EventQueue &queue); template<typename T, typename A0> friend boost::shared_ptr<event> create_event(boost::shared_ptr<T> that, typename cb1<T, A0>::type cb, EventQueue &queue); // event post friends friend void post_to_event(boost::shared_ptr<event> &ev); template<typename A0> friend void post_to_event(boost::shared_ptr<event> &ev, boost::shared_ptr<A0> &data); }; // helper functions which create new eventstemplate<typename T> boost::shared_ptr<event> create_event(boost::shared_ptr<T> that, typename cb0<T>::type cb, EventQueue &queue) { // encapsulate an object and it's member function using bind return boost::shared_ptr<event>(new event(queue, boost::bind(cb, that))); } template<typename T, typename A0> boost::shared_ptr<event> create_event(boost::shared_ptr<T> that, typename cb1<T, A0>::type cb, EventQueue &queue) { // encapsulate an object and it's member function using bind, placeholder for argument to come return boost::shared_ptr<event>(new event(queue, boost::bind(cb, that, _1))); }//----------------------------------------------------------------------- // helper functions which post events onto the event's queuevoid post_to_event(boost::shared_ptr<event> &ev) { ev->queue_.post(ev->job_); } template<typename A0>void post_to_event(boost::shared_ptr<event> &ev, boost::shared_ptr<A0> &data) { // bind the data to the job ev->queue_.post(boost::bind(ev->job_, data)); }//----------------------------------------------------------------------- On 4 June 2010 15:05, Nat Goodspeed <nat@lindenlab.com> wrote:
Nat Goodspeed wrote:
boost::signals2::slot (in boost/signals2/slot.hpp) does wrap a bind() call
in a way that you might adapt if this is what you want.
I intended to add: if you don't mind requiring a C++1x-compliant compiler, you could use perfect forwarding instead.
_______________________________________________ Boost-users mailing list Boost-users@lists.boost.org http://lists.boost.org/mailman/listinfo.cgi/boost-users
Steve Lorimer wrote:
Oooh, what's perfect forwarding?
C++1x intends to solve "the forwarding problem." Though others could give you a much clearer picture, I'm aware of two mechanisms to support this: variadic templates, allowing you to iterate through template arguments in a type-safe way, and a new parameter-passing mechanism that avoids the const ref vs. non-const ref overload explosion. I haven't played with either MSVC 10 or gcc 4.5, but I think you can turn on support for (a subset of) C++1x features in these compilers.
On 06/04/10 09:18, Steve Lorimer wrote:
Oooh, what's perfect forwarding?
Anyway, back to your previous question - no, I don't /hav/e to hide the boost::bind etc, but I think my concept will wrap up a lot of functionality rather nicely, and I don't think the calling point will look too ugly.
This is what I expect the creation of an event object to look like (using the code detailed below)
shared_ptr<event> ev = create_event<foo, bar>(obj, &foo::func, queue); // this for class *foo*, which has member *void foo::func(bar)* which we want to callback to
To post an event to be processed by another queue will look like this
post_to_event<bar>(ev, data); // posts an object *data* of type *foo* to event *ev*
So here's the code that I have come up with - doesn't compile though (of course!) - what do you think?
|// cross-thread event queue. multiple threads can post jobs, one or many threads can execute // jobs. class EventQueue { public: typedef boost::function<void()> Job;
private:
boost::mutex mtx_; boost::condition cnd_; typedef std::list<Job> Jobs; Jobs jobs_; int stop_;
// puts a job into the queue void post(Job job) { boost::mutex::scoped_lock lock(mtx_); jobs_.push_back(job); cnd_.notify_one(); }
public:
// pulls one job from the queue, returns false when stopped bool pull(Job* job) { boost::mutex::scoped_lock lock(mtx_); for(;;) { // handle spurious wake-ups while(!stop_ && jobs_.empty()) cnd_.wait(lock); if(!jobs_.empty() && 2 != stop_) { job->swap(jobs_.front()); // move efficiently, avoiding *job = jobs.front() jobs_.pop_front(); return true; } else if(stop_) { return false; } } }
// make pull() return false void stop(bool cancel_jobs) { boost::mutex::scoped_lock lock(mtx_); stop_ = 1 + cancel_jobs; // 1 - complete jobs, 2 - cancel jobs cnd_.notify_all(); }
EventQueue() : stop_() {} ~EventQueue() { this->stop(true); }
// event post friends friend void post_to_event(boost::shared_ptr<event> &ev); template<typename A0> friend void post_to_event(boost::shared_ptr<event> &ev, boost::shared_ptr<A0> &data); }; //-----------------------------------------------------------------------
// typedef generalised callback types template<class T> struct cb0 { typedef void (T::*type)(); // member function of the form 'T::fn()' };
template<class T, typename A0> struct cb1 { typedef void (T::*type)(boost::shared_ptr<A0>); // member function of the form 'T::fn(shared_ptr<A0>)' }; //----------------------------------------------------------------------- Steve,
I'm not real sure the attached would be helpful; however, I thought maybe it would because what the attached code does is curries a functor's argument (any number) and stores them into another functor which can then be called later. I admit that doesn't sound exactly like what you want, but maybe something in the attached code would be helpful. I hope so. However, it does make use of variadic templates and some code from: https://svn.boost.org/svn/boost/sandbox/variadic_templates ; so, that might not be acceptable to you. -regards Larry
On 06/04/10 09:18, Steve Lorimer wrote:
Oooh, what's perfect forwarding?
Anyway, back to your previous question - no, I don't /hav/e to hide the boost::bind etc, but I think my concept will wrap up a lot of functionality rather nicely, and I don't think the calling point will look too ugly.
This is what I expect the creation of an event object to look like (using the code detailed below)
shared_ptr<event> ev = create_event<foo, bar>(obj, &foo::func, queue); // this for class *foo*, which has member *void foo::func(bar)* which we want to callback to
To post an event to be processed by another queue will look like this
post_to_event<bar>(ev, data); // posts an object *data* of type *foo* to event *ev*
So here's the code that I have come up with - doesn't compile though (of course!) - what do you think? [snip] Steve, I tried compiling your code after slight modification to use variadic templates; however, I'm getting several minor errors. The 1st attachment is the code, the 2nd is the errors.
You might try correcting the minor errors and see if it compiles. HTH. -Larry Compilation started at Sat Jun 5 15:13:26 make run install -d `dirname /home/evansl/prog_dev/boost-svn/ro/trunk/sandbox-local/build/gcc4_4v/boost-svn/ro/sandbox/variadic_templates/libs/composite_storage/sandbox/bind_function.o` /home/evansl/download/stlfilt/gfilt -compiler:/home/evansl/download/gcc/4.4-20090630/install/bin/g++ -c -Wall -ftemplate-depth-300 -O0 -g3 -fno-inline -std=gnu++0x -DCXX0X_VARIADIC_TEMPLATES -I/home/evansl/prog_dev/boost-svn/ro/sandbox/variadic_templates -I/home/evansl/prog_dev/boost-svn/ro/sandbox/switch -I/home/evansl/prog_dev/boost-svn/ro/trunk/sandbox-local/lje -I/home/evansl/prog_dev/boost-svn/ro/trunk -DTEMPLATE_DEPTH=300 bind_function.cpp -MMD -o /home/evansl/prog_dev/boost-svn/ro/trunk/sandbox-local/build/gcc4_4v/boost-svn/ro/sandbox/variadic_templates/libs/composite_storage/sandbox/bind_function.o COMPILE.cmd=/home/evansl/download/gcc/4.4-20090630/install/bin/g++ -c -Wall -ftemplate-depth-300 -O0 -g3 -fno-inline -std=gnu++0x -DCXX0X_VARIADIC_TEMPLATES -I/home/evansl/prog_dev/boost-svn/ro/sandbox/variadic_templates -I/home/evansl/prog_dev/boost-svn/ro/sandbox/switch -I/home/evansl/prog_dev/boost-svn/ro/trunk/sandbox-local/lje -I/home/evansl/prog_dev/boost-svn/ro/trunk -DTEMPLATE_DEPTH=300 bind_function.cpp -MMD -o /home/evansl/prog_dev/boost-svn/ro/trunk/sandbox-local/build/gcc4_4v/boost-svn/ro/sandbox/variadic_templates/libs/composite_storage/sandbox/bind_function.o BD Software STL Message Decryptor v3.10 for gcc 2/3/4 bind_function.cpp:21: error: 'mutex' in namespace 'boost' does not name a type bind_function.cpp:22: error: 'condition' in namespace 'boost' does not name a type bind_function.cpp:71: error: 'event' was not declared in this scope bind_function.cpp:71: error: template argument 1 is invalid bind_function.cpp: In member function 'void EventQueue::post( boost::function<void()> )': bind_function.cpp:30: error: 'boost::mutex' has not been declared bind_function.cpp:30: error: expected ';' before 'lock' bind_function.cpp:32: error: 'cnd_' was not declared in this scope bind_function.cpp: In member function 'bool EventQueue::pull( boost::function<void()> * )': bind_function.cpp:40: error: 'boost::mutex' has not been declared bind_function.cpp:40: error: expected ';' before 'lock' bind_function.cpp:44: error: 'cnd_' was not declared in this scope bind_function.cpp:44: error: 'lock' was not declared in this scope bind_function.cpp: In member function 'void EventQueue::stop(bool)': bind_function.cpp:61: error: 'boost::mutex' has not been declared bind_function.cpp:61: error: expected ';' before 'lock' bind_function.cpp:63: error: 'cnd_' was not declared in this scope make: *** [/home/evansl/prog_dev/boost-svn/ro/trunk/sandbox-local/build/gcc4_4v/boost-svn/ro/sandbox/variadic_templates/libs/composite_storage/sandbox/bind_function.o] Error 1 Compilation exited abnormally with code 2 at Sat Jun 5 15:13:30
On 06/05/10 15:20, Larry Evans wrote: [snip]
variadic templates; however, I'm getting several minor errors. The 1st attachment is the code, the 2nd is the errors.
The 1st attachment is the corrected version. The 2nd is the Jamfile in the same directory as the source file, the 3ird is the bjam output showing it links OK. I also had to include the variadic template library headers, that was done by the Jamfile in the 4th attachment which was located in the variadic_templates directory. Is there some reason why you wouldn't want to use the variadic template compiler? HTH. -Larry project boost/composite_storage/thread : requirements <library>../../../../../trunk/libs/thread/build//boost_thread <threading>multi ; exe bind_function : bind_function.cpp ; -*- mode: compilation; default-directory: "~/prog_dev/boost-svn/ro/sandbox/variadic_templates/libs/composite_storage/sandbox/" -*- Compilation started at Tue Jun 8 15:50:25 bjam sh: icpc: not found ...patience... ...found 1184 targets... ...updating 2 targets... gcc.compile.c++ ../../../../../../../../bin.v2/prog_dev/boost-svn/ro/sandbox/variadic_templates/libs/composite_storage/sandbox/gcc-4.4_20090630/debug/threading-multi/bind_function.o gcc.link ../../../../../../../../bin.v2/prog_dev/boost-svn/ro/sandbox/variadic_templates/libs/composite_storage/sandbox/gcc-4.4_20090630/debug/threading-multi/bind_function ...updated 2 targets... Compilation finished at Tue Jun 8 15:50:35 project boost : requirements <include>. : usage-requirements <include>. ;
That is awesome - and exactly what I was looking for! Thanks Larry On 8 June 2010 21:59, Larry Evans <cppljevans@suddenlink.net> wrote:
On 06/05/10 15:20, Larry Evans wrote: [snip]
The 1st attachment is the code, the 2nd is the errors.
The 1st attachment is the corrected version. The 2nd is the Jamfile in the same directory as the source file, the 3ird is the bjam output showing it links OK. I also had to include the variadic template library headers, that was done by the Jamfile in the 4th attachment which was located in the variadic_templates
variadic templates; however, I'm getting several minor errors. directory.
Is there some reason why you wouldn't want to use the variadic template compiler?
HTH.
-Larry
project boost/composite_storage/thread : requirements <library>../../../../../trunk/libs/thread/build//boost_thread <threading>multi ;
exe bind_function : bind_function.cpp ;
-*- mode: compilation; default-directory: "~/prog_dev/boost-svn/ro/sandbox/variadic_templates/libs/composite_storage/sandbox/" -*- Compilation started at Tue Jun 8 15:50:25
bjam sh: icpc: not found ...patience... ...found 1184 targets... ...updating 2 targets... gcc.compile.c++ ../../../../../../../../bin.v2/prog_dev/boost-svn/ro/sandbox/variadic_templates/libs/composite_storage/sandbox/gcc-4.4_20090630/debug/threading-multi/bind_function.o gcc.link ../../../../../../../../bin.v2/prog_dev/boost-svn/ro/sandbox/variadic_templates/libs/composite_storage/sandbox/gcc-4.4_20090630/debug/threading-multi/bind_function ...updated 2 targets...
Compilation finished at Tue Jun 8 15:50:35
project boost : requirements <include>. : usage-requirements <include>. ;
_______________________________________________ Boost-users mailing list Boost-users@lists.boost.org http://lists.boost.org/mailman/listinfo.cgi/boost-users
Let the existing 'function' and 'bind' libraries do the work. Just make a queue of 'function<sig>' instances, by value. Use bind along with reference_wrapper and smart pointers for state, when you really want something to live outside the function instance. From: boost-users-bounces@lists.boost.org [mailto:boost-users-bounces@lists.boost.org] On Behalf Of Steve Lorimer Sent: Friday, June 04, 2010 3:34 AM To: boost-users@lists.boost.org Subject: Re: [Boost-users] [bind] & [function] How do I store member functions with generalised arguments for later invocation? Hi Nat I think we're on the same page here - but I'm looking for a more generalised form. Who owns "obj" or T that_ in my example below is unimportant - I guess we should enforce using a shared_ptr to "obj" so that it is unimportant who owns it. So this is what I came up with before your reply... and although I think it's not really the right way to do it, I think it will serve to illustrate what I'm trying to do. (this code works in conjunction with the EventQueue code I pasted in my original question (see below) // all callbacks will be of the form void T::mem_fn(...) // this for the callback of form T::mem_fn() template <typename T> class event_0 { EventQueue &queue_; // the event queue which will process the job boost::shared_ptr<T> that_; // the object whose member function the event queue will call typedef void (T::*CB_t)(); // the member function of class T which we will callback to boost::function<void()> job_; public: event_0(boost::shared_ptr<T> that, CB_t cb, EventQueue &queue) : queue_(queue), that_(that), job_(boost::bind(cb, that)) // bind the class instance and its callback function together for later invocation {} // the function the user calls which posts the job to the queue. The thread associated with the queue picks up the // job off the queue and calls it (ergo - calls that_->CB_t(); ) void post() { queue_.post(job_); } }; // So in my application I have this class: struct foo0 : boost::enable_shared_from_this<foo0> { event_0<foo0> ev; foo0(EventQueue &queue) : ev(boost::shared_ptr<foo0>(this), &foo0::bar, queue) { } void bar() { printf("%p foo::bar()\n", this); } ~foo0() { std::cout << __func__ << " - Thread id is: " << boost::this_thread::get_id() << std::endl; } }; // and I create an event which is associated with a queue. This is most likely a class member, where the class is, for eg, the producer, and the queue being passed to the event, the queue for the consumer. boost::shared_ptr<foo0> job_3(new foo0(queue)); // and then some time later, when the producer has created some data... job_3->ev.post(); // resulting in void foo0::bar() function being called on the consumer thread // so - now we want to be able to post a shared_ptr to the consumer - ie: the producer creates some data (maybe receives it from a socket) and wants to pass it to the consumer. So we want to be able to go job_4->ev.post(data); // this for the callback of form T::mem_fn(A0) template <typename T, typename A0> class event_1 { EventQueue &queue_; // the event queue which will process the job boost::shared_ptr<T> that_; // the object whose mem_fun the event queue will call typedef void (T::*CB_t)(A0); typedef boost::function<void()> Job; Job job_; public: event_1(boost::shared_ptr<T> that, CB_t cb, EventQueue &queue) : queue_(queue), that_(that), job_(boost::bind(cb, that, _1)) {} void post(A0 data) { queue_.post(boost::bind(job_, data)); } }; In other words, I hide the details of the bind from the user, and all he has to do is create some data and post it to the event, which sorts out the binding, queueing, etc - We could make it requisite that all classes which have a callback function used by the event must be derived from shared_ptr (or be shared_ptrs, or whatever), and all data posted by events ( event_1::post(A0) ) must be shared - to ensure thread safety. Ideally I'd like to be able to generalise the above somehow, and be able to cater for post(A0), post(A0, A1), post(A0, A1, A2) etc. Do you sorta see what I'm trying to do? If I can attempt to explain it better... I'm trying to create some library code which will handle inter-thread messaging in an elegant way, and hide the implementation details from the user. The idea is the EventQueue class will actually be part of an EventThread class - you create an EventThread, and all it does is endlessly loop, picking up Event objects from its EventQueue, and calling their functors (which in fact wrap up a class and member callback function and has a shared_ptr to some data which is the argument to the member callback function) - So the creation of the thread, the creation of the queue etc is hidden - all the user does is create events and call event::post(data). Do you think this is even a good design? Would you suggest a different design philosophy? TIA Steve PS: My EventQueue code is here: #include <boost/enable_shared_from_this.hpp> #include <boost/thread/thread.hpp> #include <boost/thread/mutex.hpp> #include <boost/thread/condition.hpp> #include <boost/function.hpp> #include <boost/bind.hpp> #include <stdio.h> #include <list> #include <iostream> // cross-thread event queue. multiple threads can post jobs, one or many threads can execute // jobs. class EventQueue { public: typedef boost::function<void()> Job; private: boost::mutex mtx_; boost::condition cnd_; typedef std::list<Job> Jobs; Jobs jobs_; int stop_; public: // puts a jon into the queue void post(Job job) { boost::mutex::scoped_lock lock(mtx_); jobs_.push_back(job); cnd_.notify_one(); } // pulls one job from the queue, returns false when stopped bool pull(Job* job) { boost::mutex::scoped_lock lock(mtx_); for(;;) { // handle spurious wake-ups while(!stop_ && jobs_.empty()) cnd_.wait(lock); if(!jobs_.empty() && 2 != stop_) { job->swap(jobs_.front()); // move efficiently, avoiding *job = jobs.front() jobs_.pop_front(); return true; } else if(stop_) { return false; } } } // make pull() return false void stop(bool cancel_jobs) { boost::mutex::scoped_lock lock(mtx_); stop_ = 1 + cancel_jobs; // 1 - complete jobs, 2 - cancel jobs cnd_.notify_all(); } EventQueue() : stop_() {} ~EventQueue() { this->stop(true); } }; //----------------------------------------------------------------------- // all callbacks will be of the form void T::mem_fn(...) // this for the callback of form T::mem_fn() template <typename T> class event_0 { EventQueue &queue_; // the event queue which will process the job boost::shared_ptr<T> that_; // the object whose mem_fun the event queue will call typedef void (T::*CB_t)(); typedef boost::function<void()> Job; Job job_; public: event_0(boost::shared_ptr<T> that, CB_t cb, EventQueue &queue) : queue_(queue), that_(that), job_(boost::bind(cb, that)) {} void post() { queue_.post(job_); } }; //----------------------------------------------------------------------- struct foo0 : boost::enable_shared_from_this<foo0> { event_0<foo0> ev; foo0(EventQueue &queue) : ev(boost::shared_ptr<foo0>(this), &foo0::bar, queue) { } void bar() { printf("%p foo::bar()\n", this); } ~foo0() { std::cout << __func__ << " - Thread id is: " << boost::this_thread::get_id() << std::endl; } }; //----------------------------------------------------------------------- void anotherThread(EventQueue* queue) { std::cout << __func__ << " - Thread id is: " << boost::this_thread::get_id() << std::endl; EventQueue::Job job; // wait and execute jobs till stopped while(queue->pull(&job)) job(); // execute the job } //----------------------------------------------------------------------- int main() { std::cout << __func__ << " - Thread id is: " << boost::this_thread::get_id() << std::endl; EventQueue queue; // start another thread and pass an argument boost::thread another_thread(boost::bind(anotherThread, &queue)); boost::shared_ptr<foo0> job_3(new foo0(queue)); job_3->ev.post(); // stop the queue and let it complete all jobs queue.stop(false); another_thread.join(); return 0; } //----------------------------------------------------------------------- On 3 June 2010 22:11, Nat Goodspeed <nat@lindenlab.com<mailto:nat@lindenlab.com>> wrote: it's not really clear to me whether 'obj' is owned by the consumer thread or is in the hands of the producer object. Let's consider both cases. ==== Either way: class Data { ... }; typedef boost::shared_ptr<Data> DataPtr; ==== If 'obj' is the ConsumerClass instance: class ConsumerClass { public: ... void run(); void doSomething(DataPtr data); void somethingElse(DataPtr data); ... }; typedef (ConsumerClass::*ConsumerMethod)(DataPtr); typedef boost::function<void(ConsumerClass*)> QueueItem; typedef YourFavoriteThreadSafeQueue<QueueItem> Queue; void ConsumerClass::run() { for (;;) { QueueItem item(queue.get()); item(this); } } void ProducerClass::post(ConsumerMethod method, DataPtr data) { queue.put(boost::bind(method, _1, data)); } TradeStation Group, Inc. is a publicly-traded holding company (NASDAQ GS: TRAD) of three operating subsidiaries, TradeStation Securities, Inc. (Member NYSE, FINRA, SIPC and NFA), TradeStation Technologies, Inc., a trading software and subscription company, and TradeStation Europe Limited, a United Kingdom, FSA-authorized introducing brokerage firm. None of these companies provides trading or investment advice, recommendations or endorsements of any kind. The information transmitted is intended only for the person or entity to which it is addressed and may contain confidential and/or privileged material. Any review, retransmission, dissemination or other use of, or taking of any action in reliance upon, this information by persons or entities other than the intended recipient is prohibited. If you received this in error, please contact the sender and delete the material from any computer.
For each "event" created, it will be associated with a given class instance and member function. The same event will be used over and over again to post data to the other thread. This is why I'd like to do the "setup" part first (associating the event with a given object and member function, and creating a function object of the form boost::function<void(shared_ptr<A0>)> where A0 is some argument type. Then, many times over and over during the running of the app, some data will be created (eg received from a network socket etc) and posted to the other thread. This is a real time system, so I'd like to do the minimum of configuration at run time - ergo, I'd like to have event->post(data) to just create a new function object which has the data bound to a reference to my functor which was created at construction time and which contains the details of the object and callback function. That function object is then added to the queue. On 4 June 2010 22:31, John Dlugosz <JDlugosz@tradestation.com> wrote:
Let the existing ‘function’ and ‘bind’ libraries do the work.
Just make a queue of ‘function<sig>’ instances, *by value*.
Use bind along with reference_wrapper and smart pointers for state, when you really want something to live outside the function instance.
*From:* boost-users-bounces@lists.boost.org [mailto: boost-users-bounces@lists.boost.org] *On Behalf Of *Steve Lorimer *Sent:* Friday, June 04, 2010 3:34 AM
*To:* boost-users@lists.boost.org *Subject:* Re: [Boost-users] [bind] & [function] How do I store member functions with generalised arguments for later invocation?
Hi Nat
I think we're on the same page here - but I'm looking for a more generalised form. Who owns "obj" or T that_ in my example below is unimportant - I guess we should enforce using a shared_ptr to "obj" so that it is unimportant who owns it.
So this is what I came up with before your reply... and although I think it's not really the right way to do it, I think it will serve to illustrate what I'm trying to do. (this code works in conjunction with the EventQueue code I pasted in my original question (see below)
// all callbacks will be of the form void T::mem_fn(...)
// this for the callback of form T::mem_fn()
template <typename T>
class event_0
{
EventQueue &queue_; // the event queue which will process the job
boost::shared_ptr<T> that_; // the object whose member function the event queue will call
typedef void (T::*CB_t)(); // the member function of class T which we will callback to
boost::function<void()> job_;
public:
event_0(boost::shared_ptr<T> that, CB_t cb, EventQueue &queue) :
queue_(queue),
that_(that),
job_(boost::bind(cb, that)) // bind the class instance and its callback function together for later invocation
{}
// the function the user calls which posts the job to the queue. The thread associated with the queue picks up the
// job off the queue and calls it (ergo - calls that_->CB_t(); )
void post() { queue_.post(job_); }
};
// So in my application I have this class:
struct foo0
: boost::enable_shared_from_this<foo0>
{
event_0<foo0> ev;
foo0(EventQueue &queue) : ev(boost::shared_ptr<foo0>(this), &foo0::bar, queue) { }
void bar()
{
printf("%p foo::bar()\n", this);
}
~foo0()
{
std::cout << __func__ << " - Thread id is: " << boost::this_thread::get_id() << std::endl;
}
};
*// and I create an event which is associated with a queue. This is most likely a class member, where the class is, for eg, the producer, and the queue being passed to the event, the queue for the consumer.*
boost::shared_ptr<foo0> job_3(new foo0(queue));
*// and then some time later, when the producer has created some data...*
job_3->ev.post(); // resulting in void foo0::bar() function being called on the consumer thread
*// so - now we want to be able to post a shared_ptr to the consumer - ie: the producer creates some data (maybe receives it from a socket) and wants to pass it to the consumer. So we want to be able to go job_4->ev.post(data);*
// this for the callback of form T::mem_fn(A0)
template <typename T, typename A0>
class event_1
{
EventQueue &queue_; // the event queue which will process the job
boost::shared_ptr<T> that_; // the object whose mem_fun the event queue will call
typedef void (T::*CB_t)(A0);
typedef boost::function<void()> Job;
Job job_;
public:
event_1(boost::shared_ptr<T> that, CB_t cb, EventQueue &queue) :
queue_(queue), that_(that), job_(boost::bind(cb, that, _1))
{}
void post(A0 data) { queue_.post(boost::bind(job_, data)); }
};
In other words, I hide the details of the bind from the user, and all he has to do is create some data and post it to the event, which sorts out the binding, queueing, etc - We could make it requisite that all classes which have a callback function used by the event must be derived from shared_ptr (or be shared_ptrs, or whatever), and all data posted by events ( event_1::post(A0) ) must be shared - to ensure thread safety.
Ideally I'd like to be able to generalise the above somehow, and be able to cater for post(A0), post(A0, A1), post(A0, A1, A2) etc.
Do you sorta see what I'm trying to do? If I can attempt to explain it better... I'm trying to create some library code which will handle inter-thread messaging in an elegant way, and hide the implementation details from the user. The idea is the EventQueue class will actually be part of an EventThread class - you create an EventThread, and all it does is endlessly loop, picking up Event objects from its EventQueue, and calling their functors (which in fact wrap up a class and member callback function and has a shared_ptr to some data which is the argument to the member callback function) - So the creation of the thread, the creation of the queue etc is hidden - all the user does is create events and call event::post(data).
Do you think this is even a good design? Would you suggest a different design philosophy?
TIA
Steve
PS: My EventQueue code is here:
#include <boost/enable_shared_from_this.hpp>
#include <boost/thread/thread.hpp>
#include <boost/thread/mutex.hpp>
#include <boost/thread/condition.hpp>
#include <boost/function.hpp>
#include <boost/bind.hpp>
#include <stdio.h>
#include <list>
#include <iostream>
// cross-thread event queue. multiple threads can post jobs, one or many threads can execute
// jobs.
class EventQueue
{
public:
typedef boost::function<void()> Job;
private:
boost::mutex mtx_;
boost::condition cnd_;
typedef std::list<Job> Jobs;
Jobs jobs_;
int stop_;
public:
// puts a jon into the queue
void post(Job job)
{
boost::mutex::scoped_lock lock(mtx_);
jobs_.push_back(job);
cnd_.notify_one();
}
// pulls one job from the queue, returns false when stopped
bool pull(Job* job)
{
boost::mutex::scoped_lock lock(mtx_);
for(;;)
{ // handle spurious wake-ups
while(!stop_ && jobs_.empty())
cnd_.wait(lock);
if(!jobs_.empty() && 2 != stop_)
{
job->swap(jobs_.front()); // move efficiently, avoiding *job = jobs.front()
jobs_.pop_front();
return true;
}
else if(stop_)
{
return false;
}
}
}
// make pull() return false
void stop(bool cancel_jobs)
{
boost::mutex::scoped_lock lock(mtx_);
stop_ = 1 + cancel_jobs; // 1 - complete jobs, 2 - cancel jobs
cnd_.notify_all();
}
EventQueue() : stop_() {}
~EventQueue() { this->stop(true); }
};
//-----------------------------------------------------------------------
// all callbacks will be of the form void T::mem_fn(...)
// this for the callback of form T::mem_fn()
template <typename T>
class event_0
{
EventQueue &queue_; // the event queue which will process the job
boost::shared_ptr<T> that_; // the object whose mem_fun the event queue will call
typedef void (T::*CB_t)();
typedef boost::function<void()> Job;
Job job_;
public:
event_0(boost::shared_ptr<T> that, CB_t cb, EventQueue &queue) :
queue_(queue), that_(that), job_(boost::bind(cb, that))
{}
void post() { queue_.post(job_); }
};
//-----------------------------------------------------------------------
struct foo0
: boost::enable_shared_from_this<foo0>
{
event_0<foo0> ev;
foo0(EventQueue &queue) : ev(boost::shared_ptr<foo0>(this), &foo0::bar, queue) { }
void bar()
{
printf("%p foo::bar()\n", this);
}
~foo0()
{
std::cout << __func__ << " - Thread id is: " << boost::this_thread::get_id() << std::endl;
}
};
//-----------------------------------------------------------------------
void anotherThread(EventQueue* queue)
{
std::cout << __func__ << " - Thread id is: " << boost::this_thread::get_id() << std::endl;
EventQueue::Job job;
// wait and execute jobs till stopped
while(queue->pull(&job))
job(); // execute the job
}
//-----------------------------------------------------------------------
int main()
{
std::cout << __func__ << " - Thread id is: " << boost::this_thread::get_id() << std::endl;
EventQueue queue;
// start another thread and pass an argument
boost::thread another_thread(boost::bind(anotherThread, &queue));
boost::shared_ptr<foo0> job_3(new foo0(queue));
job_3->ev.post();
// stop the queue and let it complete all jobs
queue.stop(false);
another_thread.join();
return 0;
}
//-----------------------------------------------------------------------
On 3 June 2010 22:11, Nat Goodspeed <nat@lindenlab.com> wrote:
it's not really clear to me whether 'obj' is owned by the consumer thread or is in the hands of the producer object. Let's consider both cases.
==== Either way:
class Data { ... }; typedef boost::shared_ptr<Data> DataPtr;
==== If 'obj' is the ConsumerClass instance:
class ConsumerClass { public: ... void run(); void doSomething(DataPtr data); void somethingElse(DataPtr data); ... };
typedef (ConsumerClass::*ConsumerMethod)(DataPtr);
typedef boost::function<void(ConsumerClass*)> QueueItem; typedef YourFavoriteThreadSafeQueue<QueueItem> Queue;
void ConsumerClass::run() { for (;;) { QueueItem item(queue.get()); item(this); } }
void ProducerClass::post(ConsumerMethod method, DataPtr data) { queue.put(boost::bind(method, _1, data)); }
TradeStation Group, Inc. is a publicly-traded holding company (NASDAQ GS: TRAD) of three operating subsidiaries, TradeStation Securities, Inc. (Member NYSE, FINRA, SIPC and NFA), TradeStation Technologies, Inc., a trading software and subscription company, and TradeStation Europe Limited, a United Kingdom, FSA-authorized introducing brokerage firm. None of these companies provides trading or investment advice, recommendations or endorsements of any kind. The information transmitted is intended only for the person or entity to which it is addressed and may contain confidential and/or privileged material. Any review, retransmission, dissemination or other use of, or taking of any action in reliance upon, this information by persons or entities other than the intended recipient is prohibited. If you received this in error, please contact the sender and delete the material from any computer.
_______________________________________________ Boost-users mailing list Boost-users@lists.boost.org http://lists.boost.org/mailman/listinfo.cgi/boost-users
participants (4)
-
John Dlugosz
-
Larry Evans
-
Nat Goodspeed
-
Steve Lorimer