October 2012 - Boost - lists.preview.boost.org

[thread] thread_specific_ptr leaks on VS2012 when using std::async()
by Klaim - Joël Lamotte 05 Mar '13

05 Mar '13

Hi, The title says it all. I think I found a memory leak with the help of boost.log author Andrei Semashev. I was using Boost.Log in Visual Studio 2012 (released one) and tried to use Visual Leak Detector because Visual Studio basic leak detector was reporting leaks. I isolated the leaks to usage of log and switched to VLD to check if the basic detector wasn't giving me false positive. I contacted Andrei to see if he could check it but he don't have VS2012 available so I setup some tests to isolate the problem. In the end of our exchange we concluded it was a false positive (it only leak on end of program). However it is still an annoying noise in the leak report logs, so people not understanding the source of this problem might be interested in this email. I attach the whole email exchange to this email, tell me if leak logs are necessary too. The last version of my test is at the end of the email (first program). Basically, the leak appear only when using std::async() in combination of boost::thread_specific_ptr : - using exclusively boost threading libraries (both with dynamic and static linking) result in no leak - using exclusively std threading libraries but not std::async() result in no leak - using std::async() result in a leak The second test program shows that using Intel Threading Building Blocks, I get leaks only if : - I use std::async() in combination with boost::thread_specific_ptr, as before (I mean I use tbb without a leak, then I add a std::async() call to the same task function and there is a leak) - I don't use std::async() but I don't manage the lifetime of tbb task scheduler We think the threads spawned by both tbb (that is supposed to be a superset of ppl) and std::async() (that is supposed to use ppl/concrt task scheduler) are not ended yet when the program reach the end of main (which seems logical as the task scheduler is then managed as a static global), making boost::thread_specific_ptr not release some data on thread destruction, as it is notified too late and any leak detection tool willl generate noise due to this. A way to fix this is to explicitely end the task scheduler, being tbb or ppl/concrt. I will use tbb task scheduler explicitely initialized/terminated in my project so it will solve the problem on my side. Joel Lamotte ------------------------------------ // this program was used to see the memory leak #define TEST_ASYNC // comment this to make it linear and not leak //#define WITH_BOOST_THREADS #define WITH_STD_ASYNC // uncomment this to leak //#define WITH_BOOST_LOG // boost log also leak if std::async is used #include <boost/thread.hpp> #ifdef WITH_BOOST_THREADS # include <boost/thread/future.hpp> # include <boost/chrono.hpp> # define THIS_THREAD boost::this_thread # define THREAD_LIB boost::thread # define CHRONO_LIB boost::chrono # define MY_FUTURE boost::future #else # include <future> # include <thread> # include <chrono> # define THIS_THREAD std::this_thread # define THREAD_LIB std::thread # define CHRONO_LIB std::chrono # define MY_FUTURE std::future #endif #ifdef WITH_BOOST_LOG # include <boost/log/trivial.hpp> # define MY_LOG BOOST_LOG_TRIVIAL(info) #else # include <iostream> # define MY_LOG std::cout << '\n' #endif #define BASIC_VS_LEAK_DETECTOR 0 #define VISUAL_LEAK_DETECTOR 1 #define LEAK_DETECTOR VISUAL_LEAK_DETECTOR #if LEAK_DETECTOR == BASIC_VS_LEAK_DETECTOR # define _CRTDBG_MAP_ALLOC # include <stdlib.h> # include <crtdbg.h> #else # include "vld.h" #endif struct my_data { char data[1024]; my_data() { MY_LOG << "CONSTRUCTOR my_data - " << THIS_THREAD::get_id(); } ~my_data() { MY_LOG << "DESTRUCTOR my_data - " << THIS_THREAD::get_id(); } }; boost::thread_specific_ptr< my_data > ts_ptr; void my_thread_foo() { ts_ptr.reset(new my_data()); } int work() { for( int i = 0; i < 42; ++i ) { MY_LOG << "THIS IS A MESSAGE - I = " << i; MY_LOG << "ts_ptr = " << ts_ptr.get(); my_thread_foo(); THIS_THREAD::sleep_for( CHRONO_LIB::milliseconds(10) ); } return 0; } int main(int argc, char *argv[]) { #if LEAK_DETECTOR == BASIC_VS_LEAK_DETECTOR // First simple memory leak detection for Visual Studio // use _crtBreakAlloc to put a breakpoint on the provided memory leak id allocation //_crtBreakAlloc = 1149; _CrtSetDbgFlag ( _CRTDBG_ALLOC_MEM_DF | _CRTDBG_LEAK_CHECK_DF ); #endif //int* k = new int[42]; // to check that memory leak detection does work MY_LOG << "Hello, World!"; #ifdef TEST_ASYNC # ifndef WITH_BOOST_THREADS std::vector< std::future<int> > work_in_progress(2); # ifdef WITH_STD_ASYNC for( auto& ftr : work_in_progress ) { ftr = std::async( work ); } # else for( auto& ftr : work_in_progress ) { std::packaged_task< int() > task(work); ftr = task.get_future(); std::thread thread( std::move(task) ); thread.detach(); } # endif for( auto& ftr : work_in_progress ) { ftr.get(); } # else std::vector< boost::unique_future<int> > work_in_progress(42); for( auto& ftr : work_in_progress ) { boost::packaged_task<int> task(work); // boost's packaged_task don't take the full signature as parameter, it takes the return type only ftr = task.get_future(); boost::thread thread( boost::move(task) ); thread.detach(); } for( auto& ftr : work_in_progress ) { ftr.get(); } # endif #else for( int i = 0; i < 42; ++i ) { work(); } #endif MY_LOG << "END"; return EXIT_SUCCESS; } -------------------------------------- // this program tests if the memory leaks appear with Threading Building Blocks #define BASIC_VS_LEAK_DETECTOR 0 #define VISUAL_LEAK_DETECTOR 1 #define LEAK_DETECTOR VISUAL_LEAK_DETECTOR #define WITH_ROOT_TASK //#define WITH_BOOST_LOG //#define WITH_STD_ASYNC // uncomment to leak #ifdef WITH_STD_ASYNC # include <future> #endif #include <thread> #include <atomic> #include <boost/thread.hpp> #include <tbb/tbb.h> #include <tbb/task_scheduler_init.h> #ifdef WITH_BOOST_LOG # include <boost/log/trivial.hpp> # define MY_LOG BOOST_LOG_TRIVIAL(info) #else # include <iostream> # define MY_LOG std::cout << '\n' #endif #if LEAK_DETECTOR == BASIC_VS_LEAK_DETECTOR # define _CRTDBG_MAP_ALLOC # include <stdlib.h> # include <crtdbg.h> #else # include "vld.h" #endif namespace test { struct my_data { char data[1024]; }; boost::thread_specific_ptr< my_data > ts_ptr; void my_thread_foo() { ts_ptr.reset(new my_data()); } class UpdateTask : public tbb::task { public: UpdateTask() : idx( s_idx++ ) { MY_LOG << "[UpdateTask " << idx << " ] " << std::endl; } ~UpdateTask() { MY_LOG << "[/UpdateTask " << idx << " ]" << std::endl; } tbb::task* execute() { MY_LOG << "[" << std::this_thread::get_id() << "] Tick "<< m_count <<std::endl; my_thread_foo(); ++m_count; if( m_count < 100 ) { // make sure this is re-executed this->increment_ref_count(); this->recycle_as_safe_continuation(); } return nullptr; } private: static std::atomic<int> s_idx; std::atomic<int> m_count; const int idx; }; std::atomic<int> UpdateTask::s_idx = 0; class TaskScheduler // using TBB { public: #ifdef WITH_ROOT_TASK TaskScheduler() : m_root_task( *new (tbb::task::allocate_root()) tbb::empty_task ) { m_root_task.increment_ref_count(); } ~TaskScheduler() { m_root_task.destroy(m_root_task); } #endif void register_update_task() { #ifdef WITH_ROOT_TASK m_root_task.increment_ref_count(); UpdateTask& updater = *new(m_root_task.allocate_child()) UpdateTask(); #else UpdateTask& updater = *new(tbb::task::allocate_root()) UpdateTask(); #endif m_task_list.push_back( updater ); } void run_and_wait() { #ifdef WITH_ROOT_TASK m_root_task.spawn( m_task_list ); m_root_task.wait_for_all(); #else tbb::task::spawn_root_and_wait( m_task_list ); #endif } private: tbb::task_list m_task_list; // uncomment the following line to not leak tbb //tbb::task_scheduler_init m_engine_instance; // We explicitly create and destroy the TBB task scheduler to avoid memory leak warning noises. #ifdef WITH_ROOT_TASK tbb::task& m_root_task; #endif }; void tbb_test() { TaskScheduler task_scheduler; for( int i = 0; i < 50; ++i ) { task_scheduler.register_update_task(); } task_scheduler.run_and_wait(); } } int main(int argc, char *argv[]) { #if LEAK_DETECTOR == BASIC_VS_LEAK_DETECTOR // First simple memory leak detection for Visual Studio // use _crtBreakAlloc to put a breakpoint on the provided memory leak id allocation //_crtBreakAlloc = 148; _CrtSetDbgFlag ( _CRTDBG_ALLOC_MEM_DF | _CRTDBG_LEAK_CHECK_DF ); #endif #ifdef WITH_STD_ASYNC std::atomic<bool> finished = false; std::async( [&]{ while( !finished ) { MY_LOG << "POP"; test::my_thread_foo(); std::this_thread::sleep_for( std::chrono::milliseconds(500) ); } } ); #endif test::tbb_test(); #ifdef WITH_STD_ASYNC finished = true; #endif MY_LOG << "\nEND"; std::this_thread::sleep_for( std::chrono::seconds(4) ); return 0; }

3 5

[thread] boost::thread and thread safety annotations
by Wladimir 04 Mar '13

04 Mar '13

Hello, The Clang C++ compiler project is currently working on a thread safety analyzer. By making use of so-called thread safety annotations within the source code, several problems with synchronization primitives can be found statically. For example: usage of a data structure without acquiring its guarding lock. See here for an overview of the available attributes: http://clang.llvm.org/docs/LanguageExtensions.html#threadsafety It is customary to use these attributes by defining macros, for compatibility: http://www.mail-archive.com/linuxkernelnewbies@googlegroups.com/msg01455.ht… At one time, it was also supposed to end up in G++, but this was abandoned in favor of clang (see http://gcc.gnu.org/wiki/ThreadSafetyAnnotation) But at least the annotations are fairly standardized. I think it would be useful to add these annotations to boost::thread, so that projects using that library can make use of the static checking. As I understand it, is only a matter of marking data structures that serve as lock/scoped_lock (such as boost::mutex, boost::recursive_mutex, ...), and methods that act upon the lock with the appropriate attributes. Any interest in this? Regards, Wladimir

2 1

Fwd: [Thread] Solution to conflict with MFC?
by Thorsten Ottosen 28 Feb '13

28 Feb '13

Hi Thread maintainers, Anthony speaks about a patch to make this work. Does anyone know where it is? And could it be applied? Thanks -Thorsten -------- Original Meddelelse -------- Subject: [Thread] Solution to conflict with MFC? Date: Tue, 26 Apr 2011 13:28:56 +0200 From: Christian Larsen <contact(a)dword.dk> Reply-To: boost-users(a)lists.boost.org To: boost-users(a)lists.boost.org Newsgroups: gmane.comp.lib.boost.user Hello, I know this has been asked before, but I could not find a good solution. I'm experiencing problems with an application using both MFC and Boost.Thread. Specifically the assertion ASSERT(AfxGetModuleState() != AfxGetAppModuleState()); in AfxCoreInitModule() fails. I link Boost.Thread statically into a DLL in my project. From previous posts to this list and elsewhere, e.g. http://lists.boost.org/boost-users/2009/04/46929.php, I see that the issue is caused by setting _pRawDllMain in tss_pe.cpp. Commenting this line does "solve" the problem, but I would rather not have to hack the Boost source code this way. It seems there is a patch somewhere for this problem (referred to in http://lists.boost.org/boost-users/2009/04/47015.php) but it appears it never made it into the source. Is there any chance of this issue being solved in a better way? Best regards, Christian Larsen

5 9

[Config] N3276 decltype support on VC++ 11
by Michel Morin 28 Feb '13

28 Feb '13

IIRC, Stephan said that VC++ 11 supports N3276 decltype. Shall we "undef" BOOST_NO_CXX11_DECLTYPE_N3276 for VC++ 11? This enables automatic use of decltype-based `boost::result_of`. Trivial patch attached. Regards, Michel

11 79

[multi_index] Move Semantics
by Mark Snelling 27 Feb '13

27 Feb '13

Hi, I've been a user of Boost.MultiIndex for a number of years and think it's a great library. Is there any firm plan to add move semantics? It's been noted under "Future Work" in the docs since Boost 1.32. Regards, Mark

6 10

[Lockfree] ringbuffer
by Rich E 17 Feb '13

17 Feb '13

First off, all Boost.Lockfree tests in trunk are passing for me on clang 4.0 on OS X, hurrah! Thank you Tim for working on this for so long... I'm running through the examples now, only with the outdated documentation as a guide (more than a couple years old). I cannot find the ringbuffer container - was it renamed or removed altogether during the review period? I notice spsc_queue acts awefully like a ringbuffer, and it's implementation contains many

2 6

[fixed_point] Request for interest in a binary fixed point library
by Vicente J. Botet Escriba 02 Feb '13

02 Feb '13

Hi, the recent discussion on the MultiplePrecission Arithmetic library has show that some people has its ow fixed point library. Is there an interest in a Boost library having as base http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2012/n3352.html? I have started a prototype http://svn.boost.org/svn/boost/sandbox/fixed_point (there is no doc yet but the basic goal is quite close to the N3352 C++ proposal). This prototype should allow you to play a little bit with. If yes, what would you like to be changed, added or improved in n3352? Next follows some design decisions that IMO need to be decided before hand. * Should integers and reals be represented by separated classes? * Should signed and unsigned be represented by separated classes? * Should the library use a specific representation for signed numbers (separated sign, 2-complement? Let the user choose? * Should the library provide arbitrary range and resolution and allocators? * Should the library be open to overflow and rounding or just implement some of the possible policies? and in this case which ones? * Should fixed_point be convertible to/from integer/float/double? * Could the result of an arithmetic operation have more range and resolution than his arguments? * Is there a need for a specific I/O? * is there a need for radix other than 2 (binary)? * Should the library implement the basic functions, or should it imperatively implement the C++11 math functions? Could a first version just forward to the c++11 math functions? * Should the library support just one of the know ways to name a fixed-point, a U(a,b), nQm, ...? Provide some ways to move from one to another? * Could expect the same/better performances respect to hand written code? * What should be the size used by a fixed_point instance? _fast? _least? Should the user be able to decide which approach is better for his needs? * Which should be the namespace? boost? boost/fixed_point? boost/binary_fixed_point? boost/bfp? ... * others you can think of. Please, replay if you are interested, have some experience in the domain, good ideas, .. Best regards, Vicente

6 44

[parameter] std proposal
by Lorenzo Caminiti 07 Jan '13

07 Jan '13

Hello all, Is there a C++ standard proposal for named parameter addition? I am looking for an "N-paper" for named parameters similar to N2081 for concepts, or N1962 for contracts. Thank you. -- Sent from my mobile device Lorenzo

6 8

[config] Proposed BOOST_NOEXCEPT_NOTHROW macro
by Beman Dawes 03 Jan '13

03 Jan '13

The three current BOOST_NOEXCEPT* macro variants miss the case where C++03 code has a "throw()" throw-specifier. Classes that derive from standard library exceptions need this to avoid breaking code on C++03 compilers. I'm not enamored with the BOOST_NOEXCEPT_NOTHROW name, so feel free to offer suggestions for alternate names. Here is the full set of macros: #ifdef BOOST_NO_NOEXCEPT # define BOOST_NOEXCEPT # define BOOST_NOEXCEPT_NOTHROW throw() <---- new macro # define BOOST_NOEXCEPT_IF(Predicate) # define BOOST_NOEXCEPT_EXPR(Expression) false #else # define BOOST_NOEXCEPT noexcept # define BOOST_NOEXCEPT_NOTHROW noexcept <---- new macro # define BOOST_NOEXCEPT_IF(Predicate) noexcept((Predicate)) # define BOOST_NOEXCEPT_EXPR(Expression) noexcept((Expression)) #endif Unless there are objections, I'll commit the changes Wednesday afternoon Eastern US time. --Beman

7 11

Using c++11 features in clang
by Marshall Clow 10 Dec '12

10 Dec '12

In response to a some questions on the boost-users list (and boost bug reports), I wrote up this: http://marshall.calepin.co/llvmclang-and-standard-libraries-on-mac-os-x.html I don't know if this should be a FAQ entry for Boost, or for clang. Suggestions/comments/edits welcome. -- Marshall Marshall Clow Idio Software <mailto:mclow.lists@gmail.com> A.D. 1517: Martin Luther nails his 95 Theses to the church door and is promptly moderated down to (-1, Flamebait). -- Yu Suzuki

3 3