Here is some scaled down code of generally what I'm doing:

-----------------------------------------------------------------------------------------------

boost::thread_specific_ptr <ifstream> infile;
boost::thread_specific_ptr< std::vector<std::string> > file_buffer;
boost::thread_specific_ptr<std::string> TempLine;
//more ptrs here
void ReadFile(wxString FileName)
{

     infile.reset(new ifstream);
     file_buffer.reset ( new std::vector<std::string> );
     TempLine.reset ( new std::string );

     //init other ptrs here

     //open file, etc here (@infile)

     //read contents into vector line by line
     while ( ! infile.get()->eof() )
     {
          getline(*infile,*TempLine);
          file_buffer.get()->push_back(*TempLine);
     }

     //other operations on file_buffer below ...........

}

void CallFunc()
{

    boost::filesystem::path p(TEMP_PATH); //refers to class member
    boost::filesystem::directory_iterator itr(p);
    boost::filesystem::directory_entry & entry = *itr;
    boost::filesystem::directory_iterator end_itr;

    std::vector< boost::thread* > t;

    int total_files = std::count_if( directory_iterator(p), directory_iterator(), bind( static_cast<bool(*)(const path&)>(is_regular_file), bind( &directory_entry::path, _1 ) ) );

    for (; itr!=end_itr; ++itr)
    {
         //read files, create threads
         t.push_back( new boost::thread(&ReadFile, this, entry.path().filename().string() ) ) ;
         t->join(); //app freezes here, if i detach() instead of join it's fine

         //some other stuff

     }       

     //delete thread pointers, etc ( join() before delete )

}

------------------------------------------------------------------------------------------------

Hope that helps




On Sun, Nov 24, 2013 at 12:07 PM, Igor R <boost.lists@gmail.com> wrote:
> I am using multiple threads for I/O operations. The issue I am having is
> joining these threads on the main thread. It appears that calling join() on
> a thread with TLS pointers, it results in a hang.

Could you provide a small self-contained repro, or at least show some code?


<...>
> Another thing to note is that when calling detach() on the thread, all of
> the threads execute properly and I get expected output. However, I need to
> call join() every x amount of threads due to OS limitations of how many
> threads can actually be created. I need to read about 5000 files so
> obviously I can't create 5000 threads as Windows maxes out at 700 I believe.
> I am reading files off of a RAID setup so I believe this is a situation that
> can benefit from multiple threads reading files.

Creating threads for i/o is not an efficient/scalable approach.
Consider using Boost.Asio instead (in particular, see the following
example http://www.boost.org/doc/libs/1_55_0/doc/html/boost_asio/example/cpp03/windows/transmit_file.cpp)
_______________________________________________
Boost-users mailing list
Boost-users@lists.boost.org
http://lists.boost.org/mailman/listinfo.cgi/boost-users



--
Kyle Ketterer
<reminisc3@gmail.com>
215-208-8523