Phil Endecott <spam_from_boost_dev <at> chezphil.org> writes:

I notice that Anthony Williams used the term "lockable concept" in the Boost.Threads release notes that he posted here recently. Anthony, would it be possible to say e.g. "mutex concept" in your context so that we can keep the term "lockable" available for something like this? (Or maybe they are actually the same thing. Hmmm.)

I've used the concept "lockable" to describe something which can be locked. I felt this was a better term than "mutex", since some of the things that can be locked are not mutexes. For example unique_lock<some_mutex> also models "Lockable", so you can have a unique_lock<unique_lock<some_mutex> >. Hopefully the new boost.thread docs will give you a better idea. I would not describe a pair of data+mutex as "lockable" unless you could indeed lock it and unlock it (with lock()/unlock() member functions). However, even if you could, the term "lockable" wouldn't (in my view) describe what the combined data structure was. Would "protected" better describe the intent? I know it's a keyword, but surely the idea is to protect the data from concurrent access.

Something else to consider is the interaction with atomic operations and lock-free data structures. It would be great if Locked<int> and Atomic<int>, or Locked<list<int>> and lockfree_list<int>, had very similar interfaces.

The C++0x atomic<> template will work with any type that is "trivially copy assignable and bitwise equality comparable", and it is intended that operations are protected with a mutex where they cannot be done with atomic ops. The is_lock_free() member distinguishes the two cases. I would expect this to cover many uses, though obviously you can't write atomic<list<int>>, since list<> is definitely not bitwise equality-comparable. How about synchronized<> to pinch a term from Java? Anthony -- Anthony Williams Just Software Solutions Ltd - http://www.justsoftwaresolutions.co.uk Registered in England, Company Number 5478976. Registered Office: 15 Carrallack Mews, St Just, Cornwall, TR19 7UL

Frank Mori Hess <frank.hess <at> nist.gov> writes:

The monitor and monitor_ptr classes from libpoet model the (old) Mutex concepts, with scoped_lock, scoped_try_lock, etc member types. Then they add things like supporting operator->() and operator*() on the scoped lock types, for access to the data.

Looks good. Anthony -- Anthony Williams Just Software Solutions Ltd - http://www.justsoftwaresolutions.co.uk Registered in England, Company Number 5478976. Registered Office: 15 Carrallack Mews, St Just, Cornwall, TR19 7UL

Phil Endecott <spam_from_boost_dev <at> chezphil.org> writes:

(Off-topic: in the release notes that you posted yesterday, I didn't see a sentence saying the changes are to make Boost.Threads closer to the std::thread proposal.)

You're right: there wasn't one. I'll add one.

Anthony Williams wrote:

...

I would not describe a pair of data+mutex as "lockable" unless you could indeed lock it and unlock it (with lock()/unlock() member functions).

Well you can do exactly that if you have something like

template <typename T, typename MUTEX_T = mutex> class Lockable: public T { MUTEX_t mut; public: void lock() { mut.lock(); } void unlock() { mut.unlock(); } };

Or even inherit from T and MUTEX_T.

I think I'm starting to like this idea. If Lockable<T> models your "lockable concept", then it can be used with the multi-lock algorithm and so on.

Yes.

...

However, even if you could, the term "lockable" wouldn't (in my view) describe what the combined data structure was.

I would disagree, but it just comes down to how we have individually come to use these informal terms.

Indeed. Rereading my sentence above, it is not quite as clear as I intended. Any type that implemented the appropriate interface for the concept would be "lockable" from my POV. However, "lockable" doesn't highlight for me the intent of packaging a data item with a mutex: Lockable<int> is not just a "lockable int" (i.e. "an int that can be locked") --- the intent (safe concurrent access) is more than that.

...

Would "protected" better describe the intent? I know it's a keyword, but surely the idea is to protect the data from concurrent access.

Well, "protected" doesn't actually say the "...from concurrent access" bit. It could be protected from anything. Synchronized is a bit better (though I'd need a #define to let me spell it right ). I'd still vote for Lockable though.

Naming is tricky :-). safe_for_concurrent_access<int> is a bit long-winded.

...

The C++0x atomic<> template will work with any type that is "trivially copy assignable and bitwise equality comparable", and it is intended that operations are protected with a mutex where they cannot be done with atomic ops. The is_lock_free() member distinguishes the two cases. I would expect this to cover many uses, though obviously you can't write atomic<list<int>>, since list<> is definitely not bitwise equality-comparable.

Why have the "trivially copy assignable and bitwise equality comparable" restriction, if using a mutex is an acceptable solution?

I don't know. I've been focusing on the use of specializations that really are atomic when I've been reading the papers. I'll raise the issue. Anthony -- Anthony Williams Just Software Solutions Ltd - http://www.justsoftwaresolutions.co.uk Registered in England, Company Number 5478976. Registered Office: 15 Carrallack Mews, St Just, Cornwall, TR19 7UL

Anthony Williams schrieb:

How about synchronized<> to pinch a term from Java?

sounds nice, but it suggests that access to T is already synchronized; what about synchronizable<>? Klaus

klaus triendl wrote:

Please have a look at my code, it can be found in the boost vault, name "thread_lockable.zip".

Hi Klaus, I finally got around to looking at this. Yes, it does what I expect it to. One detail I noticed was that you make the mutex mutable, so that you can lock a const object. This is something that I have worried about, because I fear that there are "gotchas". Do any experts have opinions about this? I said before that I found my own locking pointer (or locking reference) was more verbose than explicitly locking the mutex and accessing the data, and so have not used it much. I've now remembered another practical problem with it. Here's the code without Lockable<>: void start_updater(mutex& m, int& val) { // spawn a thread that periodically locks m and changes val } mutex m; array<int,100> values; for (int i=0; i<100; ++i) { start_updater(m,values[i]); } The point is that all of the values share a mutex. Our Lockable<> classes could be used here if you wanted one mutex per value, but that could be wasteful and perhaps wrong. How can we adapt Lockable<> to work in this sort of situation? I'm wondering about something like this: Lockable< array<int,100 > values; for (int i=0; i<100; ++i) { start_updater(LockableRef<int>(values,values[i]); } i.e. LockableRef contains a reference to the mutex and one element of its "parent" Lockable<>, and presents the same interface as a data-full Lockable<>. But because Lockable and LockableRef aren't actually the same type, I can't write void start_updater(Lockable<int> i); and use it with both. Is this something that you have thought about? Phil.

Frank Mori Hess wrote:

On Sunday 23 March 2008 18:47 pm, Phil Endecott wrote:

...

The point is that all of the values share a mutex. Our Lockable<> classes could be used here if you wanted one mutex per value, but that could be wasteful and perhaps wrong. How can we adapt Lockable<> to work in this sort of situation? I'm wondering about something like this:

Lockable< array<int,100 > values;

for (int i=0; i<100; ++i) { start_updater(LockableRef<int>(values,values[i]); }

i.e. LockableRef contains a reference to the mutex and one element of its "parent" Lockable<>, and presents the same interface as a data-full Lockable<>.

I think what you would want to do is just create one scoped lock, perhaps outside of the loop (a lock_acquirer in the lockable lib syntax) and use that to access the values in the array. It could just pass an ordinary reference to the start_updater function, as long as start_updater doesn't keep any persistent copies of the reference around.

No, I don't think so. You mean something equivalent to this: mutex m; array<int,100> values; { scoped_lock l(m); for (int i=0; i<100; ++i) { start_updater(LockableRef<int>(values,values[i]); } } Right? That's not the scenario that I'm trying to describe.

It could just pass an ordinary reference to the start_updater function, as long as start_updater doesn't keep any persistent copies of the reference around.

My point is that start_updater does keep a copy of the reference; from my original post: void start_updater(mutex& m, int& val) { // spawn a thread that periodically locks m and changes val } Or to be more explicit: void run_updater(mutex& m, int& val) { while (1) { sleep(1000); scoped_lock l(m); val = read_sensor(); } } void start_updater(mutex&m, int& val) { thread t(bind(run_updater,m,val)); } Phil.

Frank Mori Hess wrote:

I'm not clear on what the advantage of passing a LockableRef would be over just passing a plain reference to a Lockable?

The Lockable that I have is a Lockable< array<int,100> >. I need something like a Lockable<int> to pass to the function, because the function is only interested in part of the protected data. I want some way to reference the "parent" Lockable's mutex and a portion of its data (element of array, field of a struct etc.) Phil.

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Monday 24 March 2008 13:49 pm, Frank Mori Hess wrote:

On Monday 24 March 2008 12:13 pm, Phil Endecott wrote:

...

The Lockable that I have is a Lockable< array<int,100> >. I need something like a Lockable<int> to pass to the function, because the function is only interested in part of the protected data. I want some way to reference the "parent" Lockable's mutex and a portion of its data (element of array, field of a struct etc.)

Oh, I see now. So you would want something along the lines of a pointer/reference-like Lockable that has a constructor which can take a reference (or maybe a shared_ptr) to an already existing mutex.

Is there some way to make it possible to use boost::bind for this? So you could bind together a Lockable and a function returns a reference to the object you're interested in, and then use it later to create a scoped_lock/lock_acquirer?

Or how about an aliasing constructor for poet::monitor_ptr that works similarly to the one for shared_ptr? Using it would result in a monitor_ptr that shares its mutex (and ownership) with the original monitor_ptr, but which can have an underlying pointer of a different type and address. - -- Frank -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) iD8DBQFH5+0S5vihyNWuA4URAsMiAKCvdXcB9l5XAnYlgXuQnbgAOJxJiwCgzIpI ejhStYEhamzrNoeoPgrL0oM= =TqGK -----END PGP SIGNATURE-----

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Wednesday 02 April 2008 16:50 pm, klaus triendl wrote:

Frank Mori Hess schrieb:

...

Incidentally, I find the example usage of lock acquirers in the thread_lockable examples a bit verbose and dangerous. That is stuff like:

mbox::writelock_acquirer<mutexlockable_int> int_lock(*pLockableInt); int& i = access_acquiree(int_lock); i = 5;

Why not just support operator* (and operator-> for calling class methods) on the lock_acquirer class? Then you could do something like:

I like the less verbose method with operator*... the intent is to force a named lvalue lock_acquirer object.

...

mbox::writelock_acquirer<mutexlockable_int> int_lock(*pLockableInt); *int_lock = 5;

But what prevents me from writing: int& i = *int_lock;

Nothing, it just makes it less tempting to do so. You only need to add one character (the asterisk) to get a reference from the lock_acquirer directly. Presumably, the only reason you were creating the reference before was because it was more convenient to write i = 5; than access_acquiree(int_lock) = 5; - -- Frank -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) iD8DBQFH9OGU5vihyNWuA4URAp8DAJ0fNWckUD0ADjUfmvuS1qe8KAaJgACgkplz mHOSapSE19SN4rY0YsFGPeQ= =EYOn -----END PGP SIGNATURE-----

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Wednesday 02 April 2008 16:46 pm, klaus triendl wrote:

Phil Endecott schrieb:

...

i.e. LockableRef contains a reference to the mutex and one element of its "parent" Lockable<>, and presents the same interface as a data-full Lockable<>.

But because Lockable and LockableRef aren't actually the same type, I can't write

void start_updater(Lockable<int> i);

and use it with both.

Is this something that you have thought about?

Well, I didn't think about this one. But what do you think of the following?

<code> template<typename T_type, typename T_mutex> struct lockable //: ... { // ...

// return a lockable_ref, see below lockable_ref<std::tr1::remove_extent<T_type>, T_mutex> operator [](std::size_t idx) { return lockable_ref<std::tr1::remove_extent<T_type>, T_mutex>(access_volatile()[idx], mutex()); } };

This would only support array-like types. Are you familiar with the aliasing constructor of shared_ptr? I put something similar in poet::monitor_ptr, but it's only in cvs at the moment. Anyways, it's a more general solution to this kind of situation, although it doesn't provide much of a safeguard against misuse. Anyways, your example snippet would look something like this: <code> void start_updater(poet::monitor_ptr<int> i) { //... } poet::monitor_ptr<array<int, 100> > values; poet::monitor_ptr<int> one_value; //... for (int i=0; i<100; ++i) { start_updater(poet::monitor_ptr<int>(values, &values->at(i))); } start_updater(one_value); </code> - -- Frank -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) iD8DBQFH9N2N5vihyNWuA4URAieWAKDPq4o2VKHIoRKeiBZmz3gN6N7VIgCg5MgQ PnAdePswe+M7crNq7SLcOlk= =IC+K -----END PGP SIGNATURE-----

vicente.botet schrieb:

...

Inspired by Andrei Alexandrescu's article "volatile - Multithreaded Programmer's Best Friend" on ddj I implemented a concept similar to a locking pointer called "lock_acquirer" that collects acquisition of a mutex and a volatile cast of the locked type. It is even more threadsafe to use than a locking pointer.

I'm interested in your implementation, but much more in seen how this lock_acquirer differs from the locking_ptr and in seen how it can works with the shared mutex. Could you be more explicit about how it is more threadsafe?

There are other locking pointer like inspired also by Andrei Alexandrescu's work that will be interesting as on_derreference_locking_ptr(on_derreference_lock_acquirer) and externally_locked.

Hi Vicente, I expressed myself too vague - the lock_acquirer itself isn't more threadsafe than a locking_ptr but the way the programmer has to use it leads to more threadsafe programming behaviour, I believe. Everything depends on the usage scenario, of course. A locking_ptr takes the lockable (locker, or whatever you call it) and provides access via operator ->() and operator *(); thus you can write: <code> lockable<T, mutex> l; T* p = locking_ptr<lockable<T, mutex> >(l).operator ->(); T& o = *locking_ptr<lockable<T, mutex> >(l); </code> ... constructing the locking_ptr and accessing an object in a single line. lock_acquirer doesn't aim to be a smart pointer (though I like also the idea of having an automatic locking while accessing an object's method). It is constructed from a lockable but access to the object is only granted via a protected friend template function forcing to pass a named lock_acquirer object: <code> lockable<T, mutex> l; lock_acquirer<lockable<T, mutex> > a(l); T& o = access_acquiree(a); </code> An additional bonus of lock_acquirer is that the programmer can specify a locking policy (read/write) as a template parameter and the lock_acquirer selects an appropriate r/w lock for the given mutex type. If the locking policy is read access then lock_acquirer grants const-access only to the synchronized object even if T in lockable<T> is non-const: <code> lockable<int, mutex> l; // readlock_acquirer derives from // lock_acquirer<readlock, lockable<int, mutex> > readlock_acquirer<lockable<int, mutex> > a(l); const int& o = access_acquiree(a); </code>

...

Also, the locked type in a lockable can, if wanted, by a restricted interface only be accessed by a lock_acquirer thus forcing the programmer to a very threadsafe programming style.

Could you show us how?

The lockable type has public access methods, a safe_lockable however makes lock_acquirer a friend: template<typename T, typename T_mutex> struct safe_lockable: public lockable_base<T_mutex> { template<...> friend class lock_acquirer; protected: volatile_T& access_volatile(); T& access_nonvolatile(); }; Klaus

Hi, From: "klaus triendl" <klaus@triendl.eu>

vicente.botet schrieb: [snip]

...

I'm interested in your implementation, but much more in seen how this lock_acquirer differs from the locking_ptr and in seen how it can works with the shared mutex. Could you be more explicit about how it is more threadsafe?

There are other locking pointer like inspired also by Andrei Alexandrescu's work that will be interesting as on_derreference_locking_ptr(on_derreference_lock_acquirer) and externally_locked.

Hi Vicente,

I expressed myself too vague - the lock_acquirer itself isn't more threadsafe than a locking_ptr but the way the programmer has to use it leads to more threadsafe programming behaviour, I believe. Everything depends on the usage scenario, of course.

A locking_ptr takes the lockable (locker, or whatever you call it) and provides access via operator ->() and operator *(); thus you can write: <code> lockable<T, mutex> l; T* p = locking_ptr<lockable<T, mutex> >(l).operator ->(); T& o = *locking_ptr<lockable<T, mutex> >(l); </code>

... constructing the locking_ptr and accessing an object in a single line.

lock_acquirer doesn't aim to be a smart pointer (though I like also the idea of having an automatic locking while accessing an object's method). It is constructed from a lockable but access to the object is only granted via a protected friend template function forcing to pass a named lock_acquirer object: <code> lockable<T, mutex> l; lock_acquirer<lockable<T, mutex> > a(l); T& o = access_acquiree(a); </code>

In (*) below, is the mutex locked or not? If yes, why do we need access_acquiree(a)? If not when the mutex is locked? <code> lockable<T, mutex> l; lock_acquirer<lockable<T, mutex> > a(l); //(*) T& o = access_acquiree(a); //(**) </code> I supose that it is not possible to unlock l in (**). Your design recall the one of externally_locked class. externally_locked cloaks an object of type T and a reference of a Lockable objet, used to synchronize this object and possibily others. Actually provides full access to that object through the get member function (set could also be considered), provided you pass a reference to a Locker object that owns the Lockable. Here it is my implementation: <code> template <typename T, typename Lockable> class externally_locked { public: externally_locked(T& obj, Lockable& lockable) : obj_(obj) , lockable_(lockable) {} template <class Locker> T& get(Locker& locker) { BOOST_STATIC_ASSERT(is_strict_locker<Locker>); if (locker) { Locker try_lock(lockable_, locking_traits<Lockable>::try_to_lock()); if (!try_lock) { return obj_; } else { try_lock.unlock(); throw bad_lock(); } } else { throw bad_lock(); } } /// .... other functions private: T& obj_; Lockable& lockable; }; </code> And used as follows <code> T t; boost::mutex m; externally_locked<T, boost::mutex> el_t(t, m); { boost::lock_guard<mutex> g(l); ... /// from here can use el_t.get(g). to access 't' in a thread safe mode. el_t.get(g).fct(); } </code> is_strict_locker is true if the locker is unable to unlock the lockable instance and locking_traits provides some traits associated to the locable concept.

An additional bonus of lock_acquirer is that the programmer can specify a locking policy (read/write) as a template parameter and the lock_acquirer selects an appropriate r/w lock for the given mutex type. If the locking policy is read access then lock_acquirer grants const-access only to the synchronized object even if T in lockable<T> is non-const: <code> lockable<int, mutex> l; // readlock_acquirer derives from // lock_acquirer<readlock, lockable<int, mutex> > readlock_acquirer<lockable<int, mutex> > a(l); const int& o = access_acquiree(a); </code>

Which is the type of access_acquiree parameter? Have you un implementation using the boost shared_mutex, ?

...

...

Also, the locked type in a lockable can, if wanted, by a restricted interface only be accessed by a lock_acquirer thus forcing the programmer to a very threadsafe programming style.

Could you show us how?

The lockable type has public access methods, a safe_lockable however makes lock_acquirer a friend:

template<typename T, typename T_mutex> struct safe_lockable: public lockable_base<T_mutex> { template<...> friend class lock_acquirer;

protected: volatile_T& access_volatile(); T& access_nonvolatile(); };

Shouldn't safe_lockable use private or protected inheritance? This recalls me the backdoor pattern already used in the preceding version of the threads library. I really liked how the mutex were designed. It was really safe. No possibility to lock without unlock. The single problem was that the backdoor class was not provided as part of the interface, it was on the detail namespace. This two level interface, safe interface through the public interface and an unsafe one using a backdoor seams to me very promising. IMHO it was a pitie that the thread proposal for C++0x has removed this. I suppose that they had good raisons to remove it. Here it is the design of a mutex and a lock_guard with a backdoor. template <class Lockable> struct lockable_backdoor { lockable_backdoor(Lockable&); lock(); unlock(); Lockable lock_; }; class mutex : boost::noncopyable{ public: // no safe interface other than using guards as lock_guard, ... private: template <typename> friend class lockable_backdoor; // ... }; template<typename Lockable> class lock_guard { public: explicit lock_guard(Lockable& m): m_(m){ lockable_backdoor(m_).lock(); }; ~lock_guard() { lockable_backdoor(m_).lock(); } }; Best regards Vicente