Correct Mutext Destroy Behaviour, Pthreads and Boost

older
Resources available for testing on...

Robert Bell

1 Jun 2015 1 Jun '15

7:08 p.m.

If an object is implemented to support reference counting, and has an internal raw pthread mutex, the open group is pretty clear in the pthread_mutex_destroy doc on what different implementations must ensure (http://pubs.opengroup.org/onlinepubs/009695399/functions/pthread_mutex_destr...). Note the following statement at the bottom: "A mutex can be destroyed immediately after it is unlocked. For example, consider the following code:" obj_done(struct obj *op) { pthread_mutex_lock(&op->om); if (--op->refcnt == 0) { pthread_mutex_unlock(&op->om); (A) pthread_mutex_destroy(&op->om); (B) free(op); } else (C) pthread_mutex_unlock(&op->om); } In this case obj is reference counted and obj_done() is called whenever a reference to the object is dropped. Implementations are required to allow an object to be destroyed and freed and potentially unmapped (for example, lines A and B) immediately after the object is unlocked (line C)." On moving some of my underlying libraries to Boost, I took a look at Boost's Mutex object. The implementation of the Mutex object's destructor call does: ~mutex() { int const res = posix::pthread_mutex_destroy(&m); boost::ignore_unused(res); BOOST_ASSERT(!res); } My question centres on whether Boost's implementation violates the open group's requirement. The calls to posix::pthread_mutex_destroy (Boost's wrapper), usually perform a straight-through call to ::pthread_mutex_destroy, but it occurs inside the destructor, and so does this call occur "immediately after" an unlock by the group's definition? In straight pthreads, if I want to make sure I can rely on various implementations of the standard, I can simply do exactly what the open group says, unlock, and the next line destroy. But in the case of Boost, is it safe to do something like: mutex->unlock(); delete mutex_; One thing I did note of interest is that Apple's implementation seems to lock the mutex from within the destroy call, which is curious: int pthread_mutex_destroy(pthread_mutex_t *mutex) { int res; LOCK(mutex->lock); if (mutex->sig == _PTHREAD_MUTEX_SIG) { if (mutex->owner == (pthread_t)NULL && mutex->busy == (pthread_cond_t *)NULL) { mutex->sig = _PTHREAD_NO_SIG; res = ESUCCESS; } else res = EBUSY; } else res = EINVAL; UNLOCK(mutex->lock); return (res); } There seems, generally, to be a good deal of confusion surrounding when it is safe to destroy the resources associated with a mutex, and most people offer widely inaccurate comments based largely on opinion. My question points to whether code adjacency is a requirement of correct behaviour in some implementations, and if not, why not? If I have missed an obvious explanation somewhere, my apologies. I read through the different patterns of Boost threading listed in the documentation, but was not convinced the question was addressed. Thanks in advance…

Show replies by date

Vicente J. Botet Escriba

1 Jun 1 Jun

8:42 p.m.

Le 01/06/15 21:08, Robert Bell a écrit :

...

If an object is implemented to support reference counting, and has an internal raw pthread mutex, the open group is pretty clear in the pthread_mutex_destroy doc on what different implementations must ensure (http://pubs.opengroup.org/onlinepubs/009695399/functions/pthread_mutex_destr...).

Note the following statement at the bottom:

"A mutex can be destroyed immediately after it is unlocked. For example, consider the following code:"

obj_done(struct obj *op) { pthread_mutex_lock(&op->om); if (--op->refcnt == 0) { pthread_mutex_unlock(&op->om); (A) pthread_mutex_destroy(&op->om); (B) free(op); } else (C) pthread_mutex_unlock(&op->om); }

In this case obj is reference counted and obj_done() is called whenever a reference to the object is dropped. Implementations are required to allow an object to be destroyed and freed and potentially unmapped (for example, lines A and B) immediately after the object is unlocked (line C)."

On moving some of my underlying libraries to Boost, I took a look at Boost's Mutex object. The implementation of the Mutex object's destructor call does:

~mutex() { int const res = posix::pthread_mutex_destroy(&m); boost::ignore_unused(res); BOOST_ASSERT(!res); }

My question centres on whether Boost's implementation violates the open group's requirement. The calls to posix::pthread_mutex_destroy (Boost's wrapper), usually perform a straight-through call to ::pthread_mutex_destroy, but it occurs inside the destructor, and so does this call occur "immediately after" an unlock by the group's definition? As with pthread, It is up to the user to ensure that the mutex is unlocked when destroyed. I don't see any problem with the current implementation.

The standard contains the sentence "The behavior of a program is undefined if it destroys a mutex object owned by any thread or a thread terminates while owning a mutex object." C++ International Standard I have not found something similar in the documentation, so it should be missing.

...

In straight pthreads, if I want to make sure I can rely on various implementations of the standard, I can simply do exactly what the open group says, unlock, and the next line destroy. But in the case of Boost, is it safe to do something like:

mutex->unlock(); delete mutex_;

One thing I did note of interest is that Apple's implementation seems to lock the mutex from within the destroy call, which is curious:

int pthread_mutex_destroy(pthread_mutex_t *mutex) { int res;

LOCK(mutex->lock); if (mutex->sig == _PTHREAD_MUTEX_SIG) { if (mutex->owner == (pthread_t)NULL && mutex->busy == (pthread_cond_t *)NULL) { mutex->sig = _PTHREAD_NO_SIG; res = ESUCCESS; } else res = EBUSY; } else res = EINVAL; UNLOCK(mutex->lock); return (res); }

There seems, generally, to be a good deal of confusion surrounding when it is safe to destroy the resources associated with a mutex, and most people offer widely inaccurate comments based largely on opinion. My question points to whether code adjacency is a requirement of correct behaviour in some implementations, and if not, why not?

If I have missed an obvious explanation somewhere, my apologies. I read through the different patterns of Boost threading listed in the documentation, but was not convinced the question was addressed.

Please, could you create a ticket so that I don't forget to update the documentation? Thanks, Vicente

Gottlob Frege

8:45 p.m.

On Mon, Jun 1, 2015 at 3:08 PM, Robert Bell wrote:

...

If an object is implemented to support reference counting, and has an internal raw pthread mutex, the open group is pretty clear in the pthread_mutex_destroy doc on what different implementations must ensure (http://pubs.opengroup.org/onlinepubs/009695399/functions/pthread_mutex_destr...).

I think you are reading the spec wrong. See below.

...

Note the following statement at the bottom:

"A mutex can be destroyed immediately after it is unlocked. For example, consider the following code:"

Note the word "can" not "shall" or "must". Specs are very specific about the use of these words.

...

obj_done(struct obj *op) { pthread_mutex_lock(&op->om); if (--op->refcnt == 0) { pthread_mutex_unlock(&op->om); (A) pthread_mutex_destroy(&op->om); (B) free(op); } else (C) pthread_mutex_unlock(&op->om); }

In this case obj is reference counted and obj_done() is called whenever a reference to the object is dropped. Implementations are required to allow an object to be destroyed and freed and potentially unmapped (for example, lines A and B) immediately after the object is unlocked (line C)."

The point (I think) about the example, is that an implementation of pthread_mutex is NOT allowed to postpone work done in pthread_mutex_unlock. ie an implementation can not return immediately, and then do some of the work asynchronously, because the mutex could be gone by the time the async work was being done. So pthread_mutex_destroy cannot be "lazy".

...

On moving some of my underlying libraries to Boost, I took a look at Boost's Mutex object. The implementation of the Mutex object's destructor call does:

~mutex() { int const res = posix::pthread_mutex_destroy(&m); boost::ignore_unused(res); BOOST_ASSERT(!res); }

My question centres on whether Boost's implementation violates the open group's requirement. The calls to posix::pthread_mutex_destroy (Boost's wrapper), usually perform a straight-through call to ::pthread_mutex_destroy, but it occurs inside the destructor, and so does this call occur "immediately after" an unlock by the group's definition?

There is no violation. The call doesn't have to be "immediately after", but it can.

...

In straight pthreads, if I want to make sure I can rely on various implementations of the standard, I can simply do exactly what the open group says, unlock, and the next line destroy. But in the case of Boost, is it safe to do something like:

mutex->unlock(); delete mutex_;

One thing I did note of interest is that Apple's implementation seems to lock the mutex from within the destroy call, which is curious:

int pthread_mutex_destroy(pthread_mutex_t *mutex) { int res;

LOCK(mutex->lock); if (mutex->sig == _PTHREAD_MUTEX_SIG) { if (mutex->owner == (pthread_t)NULL && mutex->busy == (pthread_cond_t *)NULL) { mutex->sig = _PTHREAD_NO_SIG; res = ESUCCESS; } else res = EBUSY; } else res = EINVAL; UNLOCK(mutex->lock); return (res); }

There seems, generally, to be a good deal of confusion surrounding when it is safe to destroy the resources associated with a mutex, and most people offer widely inaccurate comments based largely on opinion. My question points to whether code adjacency is a requirement of correct behaviour in some implementations, and if not, why not?

There is no such thing as code adjacency nor "immediacy". The next line of code could happen an hour from the previous line (if your schedule decides that makes sense).

...

If I have missed an obvious explanation somewhere, my apologies. I read through the different patterns of Boost threading listed in the documentation, but was not convinced the question was addressed.

Thanks in advance…

_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost

Bjorn Reese

9:10 p.m.

On 06/01/2015 10:45 PM, Gottlob Frege wrote:

...

On Mon, Jun 1, 2015 at 3:08 PM, Robert Bell wrote:

...
If an object is implemented to support reference counting, and has an internal raw pthread mutex, the open group is pretty clear in the pthread_mutex_destroy doc on what different implementations must ensure (http://pubs.opengroup.org/onlinepubs/009695399/functions/pthread_mutex_destr...).

I think you are reading the spec wrong. See below.

...
Note the following statement at the bottom:

"A mutex can be destroyed immediately after it is unlocked. For example, consider the following code:"

Note the word "can" not "shall" or "must". Specs are very specific about the use of these words.

Further to the point, the quote comes from a non-normative section. The normative part only states that "Attempting to destroy a locked mutex or a mutex that is referenced [...] by another thread results in undefined behavior."

Robert Bell

9:46 p.m.

Thank you for everyone's responses, I appreciate it. Yes, the wording is very detailed and perhaps it was not clear in the manner I thought. The question then comes as to when it “is” safe to destroy a mutex when that mutex is effectively shared across threads. ###### instanceA Thread 1 refInstanceA1->retain(); // RefCnt = 1 // Spawns Thread 2, hands reference to instanceA but does not retain it. …. (a) . . . refInstanceA1->release(); ------ Thread 2 refInstanceA2->retain(); // Blocks because Thread 1 happens to be releasing… // Because Thread 1 reduces retain count to 0 before Thread 2 acquires the lock, Thread 1 proceeds to unlock and then tries to destroy. // Now Thread 2 acquires the lock and does so before the mutex delete call (as you pointed out if adjacency is not the criteria). ###### Unless Thread 1 actively retains the instance prior to handing it to Thread 2 (in (a) above), it’s not going to be safe. That means programmers have to actively remember to transfer ownership before moving to another thread. The standard does NOT (sorry for the caps) seem to indicate that if the thread which owns the lock calls destroy it’s ok. That would be nice as it would simply cause other threads waiting on it to error out in a nice way.

...

On 1 Jun 2015, at 13:45, Gottlob Frege wrote:

On Mon, Jun 1, 2015 at 3:08 PM, Robert Bell mailto:skyoyster@icloud.com> wrote:

...
If an object is implemented to support reference counting, and has an internal raw pthread mutex, the open group is pretty clear in the pthread_mutex_destroy doc on what different implementations must ensure (http://pubs.opengroup.org/onlinepubs/009695399/functions/pthread_mutex_destr... http://pubs.opengroup.org/onlinepubs/009695399/functions/pthread_mutex_destr...).

I think you are reading the spec wrong. See below.

...
Note the following statement at the bottom:

"A mutex can be destroyed immediately after it is unlocked. For example, consider the following code:"

Note the word "can" not "shall" or "must". Specs are very specific about the use of these words.

...
obj_done(struct obj *op) { pthread_mutex_lock(&op->om); if (--op->refcnt == 0) { pthread_mutex_unlock(&op->om); (A) pthread_mutex_destroy(&op->om); (B) free(op); } else (C) pthread_mutex_unlock(&op->om); }

In this case obj is reference counted and obj_done() is called whenever a reference to the object is dropped. Implementations are required to allow an object to be destroyed and freed and potentially unmapped (for example, lines A and B) immediately after the object is unlocked (line C)."

The point (I think) about the example, is that an implementation of pthread_mutex is NOT allowed to postpone work done in pthread_mutex_unlock. ie an implementation can not return immediately, and then do some of the work asynchronously, because the mutex could be gone by the time the async work was being done.

So pthread_mutex_destroy cannot be "lazy”.

...
On moving some of my underlying libraries to Boost, I took a look at Boost's Mutex object. The implementation of the Mutex object's destructor call does:

~mutex() { int const res = posix::pthread_mutex_destroy(&m); boost::ignore_unused(res); BOOST_ASSERT(!res); }

My question centres on whether Boost's implementation violates the open group's requirement. The calls to posix::pthread_mutex_destroy (Boost's wrapper), usually perform a straight-through call to ::pthread_mutex_destroy, but it occurs inside the destructor, and so does this call occur "immediately after" an unlock by the group's definition?

There is no violation. The call doesn't have to be "immediately after", but it can.

...
In straight pthreads, if I want to make sure I can rely on various implementations of the standard, I can simply do exactly what the open group says, unlock, and the next line destroy. But in the case of Boost, is it safe to do something like:

mutex->unlock(); delete mutex_;

One thing I did note of interest is that Apple's implementation seems to lock the mutex from within the destroy call, which is curious:

int pthread_mutex_destroy(pthread_mutex_t *mutex) { int res;

LOCK(mutex->lock); if (mutex->sig == _PTHREAD_MUTEX_SIG) { if (mutex->owner == (pthread_t)NULL && mutex->busy == (pthread_cond_t *)NULL) { mutex->sig = _PTHREAD_NO_SIG; res = ESUCCESS; } else res = EBUSY; } else res = EINVAL; UNLOCK(mutex->lock); return (res); }

There seems, generally, to be a good deal of confusion surrounding when it is safe to destroy the resources associated with a mutex, and most people offer widely inaccurate comments based largely on opinion. My question points to whether code adjacency is a requirement of correct behaviour in some implementations, and if not, why not?

There is no such thing as code adjacency nor "immediacy". The next line of code could happen an hour from the previous line (if your schedule decides that makes sense).

...
If I have missed an obvious explanation somewhere, my apologies. I read through the different patterns of Boost threading listed in the documentation, but was not convinced the question was addressed.

Thanks in advance…

_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost http://lists.boost.org/mailman/listinfo.cgi/boost

_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost http://lists.boost.org/mailman/listinfo.cgi/boost

Gottlob Frege

10:18 p.m.

On Mon, Jun 1, 2015 at 5:46 PM, Robert Bell wrote:

...

Thank you for everyone's responses, I appreciate it. Yes, the wording is very detailed and perhaps it was not clear in the manner I thought.

The question then comes as to when it “is” safe to destroy a mutex when that mutex is effectively shared across threads.

######

instanceA

Thread 1 refInstanceA1->retain();

// RefCnt = 1 // Spawns Thread 2, hands reference to instanceA but does not retain it. …. (a) . . . refInstanceA1->release();

------

Thread 2 refInstanceA2->retain(); // Blocks because Thread 1 happens to be releasing…

// Because Thread 1 reduces retain count to 0 before Thread 2 acquires the lock, Thread 1 proceeds to unlock and then tries to destroy. // Now Thread 2 acquires the lock and does so before the mutex delete call (as you pointed out if adjacency is not the criteria).

######

Unless Thread 1 actively retains the instance prior to handing it to Thread 2 (in (a) above), it’s not going to be safe. That means programmers have to actively remember to transfer ownership before moving to another thread.

Absolutely correct. Same as for other resources, not just mutexes. If you use things like smart pointers, it is typically handled for you automatically.

...

The standard does NOT (sorry for the caps) seem to indicate that if the thread which owns the lock calls destroy it’s ok.

If fact it says you can't own it when destroying - no one can own the lock (or be waiting on it) while it is being destroyed.

...

That would be nice as it would simply cause other threads waiting on it to error out in a nice way.

If one thread destroys the lock, then the other threads are waiting on nothing. There is no longer an 'it' to be waiting on. (And if the mutex uses internal allocation or other shared resources, the memory that was once a mutex could already be reused as something else) Tony

...

...
On 1 Jun 2015, at 13:45, Gottlob Frege wrote:

On Mon, Jun 1, 2015 at 3:08 PM, Robert Bell mailto:skyoyster@icloud.com> wrote:

...
If an object is implemented to support reference counting, and has an internal raw pthread mutex, the open group is pretty clear in the pthread_mutex_destroy doc on what different implementations must ensure (http://pubs.opengroup.org/onlinepubs/009695399/functions/pthread_mutex_destr... http://pubs.opengroup.org/onlinepubs/009695399/functions/pthread_mutex_destr...).

I think you are reading the spec wrong. See below.

...
Note the following statement at the bottom:

"A mutex can be destroyed immediately after it is unlocked. For example, consider the following code:"

Note the word "can" not "shall" or "must". Specs are very specific about the use of these words.

...
obj_done(struct obj *op) { pthread_mutex_lock(&op->om); if (--op->refcnt == 0) { pthread_mutex_unlock(&op->om); (A) pthread_mutex_destroy(&op->om); (B) free(op); } else (C) pthread_mutex_unlock(&op->om); }

In this case obj is reference counted and obj_done() is called whenever a reference to the object is dropped. Implementations are required to allow an object to be destroyed and freed and potentially unmapped (for example, lines A and B) immediately after the object is unlocked (line C)."

The point (I think) about the example, is that an implementation of pthread_mutex is NOT allowed to postpone work done in pthread_mutex_unlock. ie an implementation can not return immediately, and then do some of the work asynchronously, because the mutex could be gone by the time the async work was being done.

So pthread_mutex_destroy cannot be "lazy”.

...
On moving some of my underlying libraries to Boost, I took a look at Boost's Mutex object. The implementation of the Mutex object's destructor call does:

~mutex() { int const res = posix::pthread_mutex_destroy(&m); boost::ignore_unused(res); BOOST_ASSERT(!res); }

My question centres on whether Boost's implementation violates the open group's requirement. The calls to posix::pthread_mutex_destroy (Boost's wrapper), usually perform a straight-through call to ::pthread_mutex_destroy, but it occurs inside the destructor, and so does this call occur "immediately after" an unlock by the group's definition?

There is no violation. The call doesn't have to be "immediately after", but it can.

...
In straight pthreads, if I want to make sure I can rely on various implementations of the standard, I can simply do exactly what the open group says, unlock, and the next line destroy. But in the case of Boost, is it safe to do something like:

mutex->unlock(); delete mutex_;

One thing I did note of interest is that Apple's implementation seems to lock the mutex from within the destroy call, which is curious:

int pthread_mutex_destroy(pthread_mutex_t *mutex) { int res;

LOCK(mutex->lock); if (mutex->sig == _PTHREAD_MUTEX_SIG) { if (mutex->owner == (pthread_t)NULL && mutex->busy == (pthread_cond_t *)NULL) { mutex->sig = _PTHREAD_NO_SIG; res = ESUCCESS; } else res = EBUSY; } else res = EINVAL; UNLOCK(mutex->lock); return (res); }

There seems, generally, to be a good deal of confusion surrounding when it is safe to destroy the resources associated with a mutex, and most people offer widely inaccurate comments based largely on opinion. My question points to whether code adjacency is a requirement of correct behaviour in some implementations, and if not, why not?

There is no such thing as code adjacency nor "immediacy". The next line of code could happen an hour from the previous line (if your schedule decides that makes sense).

...
If I have missed an obvious explanation somewhere, my apologies. I read through the different patterns of Boost threading listed in the documentation, but was not convinced the question was addressed.

Thanks in advance…

_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost http://lists.boost.org/mailman/listinfo.cgi/boost

_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost http://lists.boost.org/mailman/listinfo.cgi/boost

_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost

3450

Age (days ago)

3450

Last active (days ago)

List overview

Download

5 comments

4 participants

participants (4)

Bjorn Reese
Gottlob Frege
Robert Bell
Vicente J. Botet Escriba