Threading question
I have the following code and it terminates as expected. When I call the destructor it interrupts the thread. But if I change the this_thread::sleep to this_thread::yield the join() never happens and the program hangs. I must be misunderstanding yield. But I really don't want a sleep if possible. Thanks, John MessageQueueHandler::~MessageQueueHandler() { // Terminate the thread pMessageQueueInputThread->interrupt(); pMessageQueueInputThread->join(); delete pMessageQueueInputThread; } void MessageQueueHandler::MessageQueueInputThread() { unsigned int receivedSize; unsigned int priority; char buffer[queueMaximumMessageSize]; boost::this_thread::interruption_enabled(); try { while(1) { // Process all messages in the queue while(inputMessageQueue->try_receive(buffer, queueMaximumMessageSize, receivedSize, priority)) { BaseMessage * msg = reinterpret_cast<BaseMessage*>(buffer); callbackTable->DispatchMessage(msg); boost::this_thread::interruption_point(); } // Give other threads a chance to run // This is also an interruption point boost::this_thread::sleep(boost::posix_time::millisec(50)); <==Here } } catch(boost::thread_interrupted const&) { return; } } ___________________________________ John Davies Contractor Home Respiratory Care Philips Home Healthcare Solutions 1740 Golden Mile Highway Monroeville, PA 15146 Email: john.davies@philips.com Fax: 724-387-4109 ________________________________ The information contained in this message may be confidential and legally protected under applicable law. The message is intended solely for the addressee(s). If you are not the intended recipient, you are hereby notified that any use, forwarding, dissemination, or reproduction of this message is strictly prohibited and may be unlawful. If you are not the intended recipient, please contact the sender by return e-mail and destroy all copies of the original message.
On 9/27/2013 11:16 AM, Quoth Davies, John:
I have the following code and it terminates as expected. When I call the destructor it interrupts the thread.
But if I change the this_thread::sleep to this_thread::yield the join() never happens and the program hangs.
I must be misunderstanding yield. But I really don’t want a sleep if possible.
The main thing to remember about yield() is that it's permitted to be a no-op if the OS feels like it, and in particular on single-core systems it will completely block lower priority threads from running. Also note that unlike sleep, yield is not listed as an interruption point. So when you're using yield you're basically setting up a 100% CPU loop that can't be interrupted, which is why it's hanging. You *could* fix part of that by adding an explicit interruption point, but that's not the ideal solution. Given that it looks like you basically want to block until there's work to do in the queue, you might want to consider using a mutex and condition_variable (notifying the condition variable when you push something into the queue, and waiting on it when the queue is empty). You're going to need a mutex on the queue operations anyway unless you're using something that's internally thread-safe. You also might want to consider using something like Boost.Asio instead. Its io_service allows you to queue arbitrary function objects to a specific thread (or threadpool) in a thread-safe manner, even if you don't want to use the actual I/O parts.
The farther I dig into Boost Interprocess the unhappier I become. Every wait seems to come down to a tight loop that is polling a condition. I'm looking into condition_variables and Asio to see if it's more of the same. ___________________________________ John Davies Contractor Home Respiratory Care Philips Home Healthcare Solutions 1740 Golden Mile Highway Monroeville, PA 15146 Email: john.davies@philips.com Fax: 724-387-4109 -----Original Message----- From: Boost-users [mailto:boost-users-bounces@lists.boost.org] On Behalf Of Gavin Lambert Sent: Thursday, September 26, 2013 7:49 PM To: boost-users@lists.boost.org Subject: Re: [Boost-users] Threading question On 9/27/2013 11:16 AM, Quoth Davies, John:
I have the following code and it terminates as expected. When I call the destructor it interrupts the thread.
But if I change the this_thread::sleep to this_thread::yield the join() never happens and the program hangs.
I must be misunderstanding yield. But I really don't want a sleep if possible.
The main thing to remember about yield() is that it's permitted to be a no-op if the OS feels like it, and in particular on single-core systems it will completely block lower priority threads from running. Also note that unlike sleep, yield is not listed as an interruption point. So when you're using yield you're basically setting up a 100% CPU loop that can't be interrupted, which is why it's hanging. You *could* fix part of that by adding an explicit interruption point, but that's not the ideal solution. Given that it looks like you basically want to block until there's work to do in the queue, you might want to consider using a mutex and condition_variable (notifying the condition variable when you push something into the queue, and waiting on it when the queue is empty). You're going to need a mutex on the queue operations anyway unless you're using something that's internally thread-safe. You also might want to consider using something like Boost.Asio instead. Its io_service allows you to queue arbitrary function objects to a specific thread (or threadpool) in a thread-safe manner, even if you don't want to use the actual I/O parts. _______________________________________________ Boost-users mailing list Boost-users@lists.boost.org http://lists.boost.org/mailman/listinfo.cgi/boost-users ________________________________ The information contained in this message may be confidential and legally protected under applicable law. The message is intended solely for the addressee(s). If you are not the intended recipient, you are hereby notified that any use, forwarding, dissemination, or reproduction of this message is strictly prohibited and may be unlawful. If you are not the intended recipient, please contact the sender by return e-mail and destroy all copies of the original message.
2013-09-27 01:48, Gavin Lambert skrev:
On 9/27/2013 11:16 AM, Quoth Davies, John:
I have the following code and it terminates as expected. When I call the destructor it interrupts the thread.
But if I change the this_thread::sleep to this_thread::yield the join() never happens and the program hangs.
I must be misunderstanding yield. But I really don’t want a sleep if possible.
The main thing to remember about yield() is that it's permitted to be a no-op if the OS feels like it, and in particular on single-core systems it will completely block lower priority threads from running.
Also note that unlike sleep, yield is not listed as an interruption point. So when you're using yield you're basically setting up a 100% CPU loop that can't be interrupted, which is why it's hanging. You *could* fix part of that by adding an explicit interruption point, but that's not the ideal solution.
Given that it looks like you basically want to block until there's work to do in the queue, you might want to consider using a mutex and condition_variable (notifying the condition variable when you push something into the queue, and waiting on it when the queue is empty). You're going to need a mutex on the queue operations anyway unless you're using something that's internally thread-safe.
You also might want to consider using something like Boost.Asio instead. Its io_service allows you to queue arbitrary function objects to a specific thread (or threadpool) in a thread-safe manner, even if you don't want to use the actual I/O parts.
Having problems with a three thread program. The application is using Boost 1.54 on a Beaglebone Black ARM target. using a single core Cortex-A8. The first thread reads data from a serial port, and store a record with the info in a list, The second thread reads from the list, and upioads data from the list to a webserver. The list is protected by a boost::mutex Queue_Lock; The third thread initializes the two first two threads and the ends up executing io_service.run(); io_service is related to an UDP based debug channel using boost::asio. By calling DebugPort.WriteLine you send a string to the UDP debug channel. The UDP debug channel, is protected by another mutex, since both main threads will use it. THREAD_WRITE is a conditional DebugPort.WriteLine, and normally this is a NOP. The second thread calls a number of routines which tries to allocate a mutex, and if not, it will yield. It looks like the second thread hangs in one of these routines. This does not occur frequently. The program was running for a week before it happened the last time. There is not enough debug prints right now to determine why, and since the application was built using a cross compiler, there is not a lot of symbols available. When attaching using gdb, we saw that it was in a mutex wait in the libpthread library Typical code. bool Server::QueueEmpty(void) { bool empty; try { THREAD_WRITE("Checking SendQueue", YELLOW); empty = true; while (1) { boost::mutex::scoped_lock lock(Queue_Lock, boost::try_to_lock); if (lock) { empty = SendQueue.empty(); break; } else { // Didn't get lock, retry after yield THREAD_WRITE("Sendstart Yielding QueueEmpty", RED); this_thread::yield(); THREAD_WRITE("Sendstart Returning QueueEmpty", RED); } } } catch (std::exception &e) { DebugPort.WriteLine(e.what(), RED); empty = true; } return empty; } Is there an obvious problem with this code? It is using Yield, which I assumed was the right way to release the CPU. Using sleep feels wrong, since I do not want to sleep, I want to wait for the mutex. The first thread implements the mutex in the same way, although not in a subroutine. Am I correct in assuming, that since I use scoped_lock, the mutex will always be released when I exit the while loop with "break;"? Best Regars Ulf Samuelsson
_______________________________________________ Boost-users mailing list Boost-users@lists.boost.org http://lists.boost.org/mailman/listinfo.cgi/boost-users
On Fri, Dec 13, 2013 at 9:25 AM, Ulf Samuelsson <boost-user@emagii.com>wrote:
Typical code.
bool Server::QueueEmpty(void) { bool empty; try { THREAD_WRITE("Checking SendQueue", YELLOW); empty = true; while (1) { boost::mutex::scoped_lock lock(Queue_Lock, boost::try_to_lock); if (lock) { empty = SendQueue.empty(); break; } else { // Didn't get lock, retry after yield THREAD_WRITE("Sendstart Yielding QueueEmpty", RED); this_thread::yield(); THREAD_WRITE("Sendstart Returning QueueEmpty", RED); } } } catch (std::exception &e) { DebugPort.WriteLine(e.what(), RED); empty = true; } return empty; }
Is there an obvious problem with this code?
- Asking if a multithreaded queue is empty is typically useless. Whatever value it returns is already out of date. - _trying_ to grab a lock, then yielding, then retrying, can be written much shorter: // get the lock, (which might require waiting/yielding until you can get it) boost::mutex::scoped_lock lock(Queue_lock); return SendQueue.empty();
It is using Yield, which I assumed was the right way to release the CPU. Using sleep feels wrong, since I do not want to sleep, I want to wait for the mutex.
If you want to wait for the mutex, wait for the mutex. Pass it to scoped_lock and it will wait for you.
The first thread implements the mutex in the same way, although not in a subroutine.
If all your code uses mutexes similarly, well, I think you are fundamentally using mutexes wrong, so the bug could be anywhere. I mean, your example code above isn't technically wrong (that I can see), but it just isn't how mutexes are meant to be used. What you are doing is overly complicated, and complicated tends to lead to bugs. Keep it simple.
Am I correct in assuming, that since I use scoped_lock, the mutex will always be released when I exit the while loop with "break;"?
yes. Even with an exception thrown, it will always release the lock. Tony
2013-12-13 18:51, Gottlob Frege skrev:
On Fri, Dec 13, 2013 at 9:25 AM, Ulf Samuelsson <boost-user@emagii.com <mailto:boost-user@emagii.com>> wrote:
Typical code.
bool Server::QueueEmpty(void) { bool empty; try { THREAD_WRITE("Checking SendQueue", YELLOW); empty = true; while (1) { boost::mutex::scoped_lock lock(Queue_Lock, boost::try_to_lock); if (lock) { empty = SendQueue.empty(); break; } else { // Didn't get lock, retry after yield THREAD_WRITE("Sendstart Yielding QueueEmpty", RED); this_thread::yield(); THREAD_WRITE("Sendstart Returning QueueEmpty", RED); } } } catch (std::exception &e) { DebugPort.WriteLine(e.what(), RED); empty = true; } return empty; }
Is there an obvious problem with this code?
- Asking if a multithreaded queue is empty is typically useless. Whatever value it returns is already out of date.
No, because this is a producer / consumer scenario. The producer will only add to the queue. The consumer is the only thread removing stuff from the queue, and is also the only thread to call QueueEmpty. If the Queue tests to be non-empty, then it will also be non-empty when the the returned value will be used. If the Queue tests to be empty, and a value is inserted between the test, and the use of the returned value, it is not a problem. since the recently added values will be sent the next time through the loop. The main loop loop empty = QueueEmpty(); ... if (not empty and other conditions) prepare data send data to webserver remove stuff form the Queue end if; Debug Output end loop;
- _trying_ to grab a lock, then yielding, then retrying, can be written much shorter:
// get the lock, (which might require waiting/yielding until you can get it) boost::mutex::scoped_lock lock(Queue_lock); return SendQueue.empty();
Yes, but I would like to understand more about the run time behaviour, and the code will allow the Debug Output to be enabled.
It is using Yield, which I assumed was the right way to release the CPU. Using sleep feels wrong, since I do not want to sleep, I want to wait for the mutex.
If you want to wait for the mutex, wait for the mutex. Pass it to scoped_lock and it will wait for you.
The first thread implements the mutex in the same way, although not in a subroutine.
If all your code uses mutexes similarly, well, I think you are fundamentally using mutexes wrong, so the bug could be anywhere. I mean, your example code above isn't technically wrong (that I can see), but it just isn't how mutexes are meant to be used. What you are doing is overly complicated, and complicated tends to lead to bugs. Keep it simple.
I agree in principle, but as I am trying to figure out what goes wrong, I need debug printouts and this is what I ended up with. Before I had one biocking mutex test in the consumer, and one in the producer, in the beginning of the loop. Then the application hung within hours. Now it took a week for it to hang.
Am I correct in assuming, that since I use scoped_lock, the mutex will always be released when I exit the while loop with "break;"?
yes. Even with an exception thrown, it will always release the lock.
Tony
_______________________________________________ Boost-users mailing list Boost-users@lists.boost.org http://lists.boost.org/mailman/listinfo.cgi/boost-users
participants (4)
-
Davies, John -
Gavin Lambert -
Gottlob Frege -
Ulf Samuelsson