Re: [Boost-users] [boost-TLS] Thread Local Storage (TLS) Question

24 Nov 2013


      Here is some scaled down code of generally what I'm doing:

-----------------------------------------------------------------------------------------------

boost::thread_specific_ptr <ifstream> infile;
boost::thread_specific_ptr< std::vectorstd::string > file_buffer;
boost::thread_specific_ptrstd::string TempLine;
//more ptrs here
void ReadFile(wxString FileName)
{

     infile.reset(new ifstream);
     file_buffer.reset ( new std::vectorstd::string );
     TempLine.reset ( new std::string );

     //init other ptrs here

     //open file, etc here (@infile)

     //read contents into vector line by line
     while ( ! infile.get()->eof() )
     {
          getline(*infile,*TempLine);
          file_buffer.get()->push_back(*TempLine);
     }

     //other operations on file_buffer below ...........

}

void CallFunc()
{

    boost::filesystem::path p(TEMP_PATH); //refers to class member
    boost::filesystem::directory_iterator itr(p);
    boost::filesystem::directory_entry & entry = *itr;
    boost::filesystem::directory_iterator end_itr;

    std::vector< boost::thread* > t;

    int total_files = std::count_if( directory_iterator(p),
directory_iterator(), bind( static_cast(is_regular_file), bind( &directory_entry::path, _1 ) ) );

    for (; itr!=end_itr; ++itr)
    {
         //read files, create threads
         t.push_back( new boost::thread(&ReadFile, this,
entry.path().filename().string() ) ) ;
         t->join(); //app freezes here, if i detach() instead of join it's
fine

         //some other stuff

     }

     //delete thread pointers, etc ( join() before delete )

}

------------------------------------------------------------------------------------------------

Hope that helps


On Sun, Nov 24, 2013 at 12:07 PM, Igor R  wrote:
...
...
I am using multiple threads for I/O operations. The issue I am having is
joining these threads on the main thread. It appears that calling join()
on
a thread with TLS pointers, it results in a hang.
Could you provide a small self-contained repro, or at least show some code?
...
Another thing to note is that when calling detach() on the thread, all of
the threads execute properly and I get expected output. However, I need
to
call join() every x amount of threads due to OS limitations of how many
threads can actually be created. I need to read about 5000 files so
obviously I can't create 5000 threads as Windows maxes out at 700 I
believe.
I am reading files off of a RAID setup so I believe this is a situation
<...>
that
...
can benefit from multiple threads reading files.
Creating threads for i/o is not an efficient/scalable approach.
Consider using Boost.Asio instead (in particular, see the following
example
http://www.boost.org/doc/libs/1_55_0/doc/html/boost_asio/example/cpp03/windo...
)
_______________________________________________
Boost-users mailing list
Boost-users@lists.boost.org
http://lists.boost.org/mailman/listinfo.cgi/boost-users
-- 
Kyle Ketterer

215-208-8523