April 2005 - Boost-users - lists.preview.boost.org

RE: [Boost-users] RE: Boost thread problem
by Sliwa, Przemyslaw (London) 14 Apr '05

14 Apr '05

It seems the usage of the vector cases problems with the multithreaded programs on the multiprocessor machines. I have changed it to queue and the bug has been resolved. I cannot reproduce it now. Thanks for help, Pshemek -----Original Message----- From: boost-users-bounces(a)lists.boost.org [mailto:boost-users-bounces@lists.boost.org] On Behalf Of C. Michailidis Sent: 14 April 2005 07:26 To: boost-users(a)lists.boost.org Subject: [Boost-users] RE: Boost thread problem > All, > > I have got a problem with the boost::thread package. > My idea was to facilitate the multiprocessor ability of my computer and > write a program which creates 10 threads which firstly write to the disk > certain amount of information and then they read it back and push back > certain objects. I am getting some memory access problems when pushing > back the objects to the global vector. Can anyone of you understand why > there is a problem? I really do not get it since the mutex is locked and > the access to the vector is exclusively guaranteed to the active thread. > > > Can you help? I tried your code on my system and it seems to work as expected. No exceptions were thrown, there was a file called 'file.txt' in the pwd, and TradeVector.size was 80000 after the run. Of course I needed to remove the include of proprietary code - it was replaced with a dummy/stub class like this one: class CFXTrade { public: CFXTrade() { }; CFXTrade( int id, string name, double amt ) { m_TradeID = id; m_Name = name; m_Amount = amt; }; int m_TradeID; string m_Name; double m_Amount; }; My system looks like this: FreeBSD medea.digitalescorts.net 5.4-PRERELEASE FreeBSD 5.4-PRERELEASE #0: Thu Mar 31 02:47:38 EST 2005 root@medea.digitalescorts.net:/usr/obj/usr/src/sys/MEDEA i386 and I have boost-1.32.0_2 libraries installed. I built the program from the command-line using the following command: "gcc hello.cpp -ggdb -I/usr/local/include -L/usr/local/lib -lboost_thread -lc_r" The problem may be platform specific. Are you linking against a thread-safe CRT? Can you provide details regarding your operating system? Which version of the boost library do you have installed? My suspicion is that you may not be using a thread-safe CRT. Of course the boost_thread library may be broken for your specific platform (but I doubt it). Sincerely, Constantino Michailidis Balston Research LLC _______________________________________________ Boost-users mailing list Boost-users(a)lists.boost.org http://lists.boost.org/mailman/listinfo.cgi/boost-users -------------------------------------------------------- If you are not an intended recipient of this e-mail, please notify the sender, delete it and do not read, act upon, print, disclose, copy, retain or redistribute it. Click here for important additional terms relating to this e-mail. http://www.ml.com/email_terms/ --------------------------------------------------------

2 1

uBLAS and stl algorithms
by Pascal Bauer 14 Apr '05

14 Apr '05

Hi List Members I have data stored in an matrix. Now, i want to acces the data of the matrix row-wise. because i want to access it in a sorted manner (without moving the matrix entries around), i make a stl-vector of matrix_rows. This works fine. But if i now use the stl algorithms to sort the stl-vector by a certain criteria, then it happens, that the algorithm is making several copies of a singe vector entry and overwriting other entries. Can anyone give me a tipp, what i'm doing wrong? ---------------------------- Here is an example code: #include<iostream> #include<vector> #include<algorithm> #include<cmath> #include<boost/numeric/ublas/matrix.hpp> #include<boost/numeric/ublas/io.hpp> namespace num=boost::numeric::ublas; template <int N> class Ordering { public: bool operator()(num::matrix_row<num::matrix<double> > A, num::matrix_row<num::matrix<double> > B) { if (A(N) < B(N) ) return true; else return false; } }; int main(int argc, char* argv[]) { const int N = 10000; const int M = 4; num::matrix<double> Matrix(N, M); for (int i = 0; i < Matrix.size1(); i++) { for (int j = 0; j < Matrix.size2(); j++) { Matrix(i, j) = drand48(); } } std::vector<num::matrix_row<num::matrix<double> > > MyAccess; for (int i = 0; i < Matrix.size1(); i++) { MyAccess.push_back(num::matrix_row<num::matrix<double> >(Matrix, i) ); } for (int i = 0; i < Matrix.size1(); i++) { std::cout << MyAccess[i] << std::endl; } std::sort(MyAccess.begin(), MyAccess.end(), Ordering<0>() ); for (int i = 0; i < Matrix.size1(); i++) { std::cout << MyAccess[i] << std::endl; } }

1 0

RE: [Boost-users] RE: Boost thread problem
by Sliwa, Przemyslaw (London) 14 Apr '05

14 Apr '05

Well, I have tried it on three systems: 1 multiprocessor Win XP -> fails, some problems with the heap, 2. single processor Win 2000 -> works fine, no problems at all, all the treads exit properly, 3. single processor Linux Suse 9.1, gcc 3.3 -> fails, segmentation fault. I am also using the boost 1.32 library. Similar compilation on Linux except -lc_r. What is it for? How to check if I am linking against thread safe libraries? On Windows the NET 2003 compiler was used, the multithreaded support was on, so I have no idea how to link against tread safe libs. Can you advise? Thanks in advance Pshemek -----Original Message----- From: boost-users-bounces(a)lists.boost.org [mailto:boost-users-bounces@lists.boost.org] On Behalf Of C. Michailidis Sent: 14 April 2005 07:26 To: boost-users(a)lists.boost.org Subject: [Boost-users] RE: Boost thread problem > All, > > I have got a problem with the boost::thread package. > My idea was to facilitate the multiprocessor ability of my computer and > write a program which creates 10 threads which firstly write to the disk > certain amount of information and then they read it back and push back > certain objects. I am getting some memory access problems when pushing > back the objects to the global vector. Can anyone of you understand why > there is a problem? I really do not get it since the mutex is locked and > the access to the vector is exclusively guaranteed to the active thread. > > > Can you help? I tried your code on my system and it seems to work as expected. No exceptions were thrown, there was a file called 'file.txt' in the pwd, and TradeVector.size was 80000 after the run. Of course I needed to remove the include of proprietary code - it was replaced with a dummy/stub class like this one: class CFXTrade { public: CFXTrade() { }; CFXTrade( int id, string name, double amt ) { m_TradeID = id; m_Name = name; m_Amount = amt; }; int m_TradeID; string m_Name; double m_Amount; }; My system looks like this: FreeBSD medea.digitalescorts.net 5.4-PRERELEASE FreeBSD 5.4-PRERELEASE #0: Thu Mar 31 02:47:38 EST 2005 root@medea.digitalescorts.net:/usr/obj/usr/src/sys/MEDEA i386 and I have boost-1.32.0_2 libraries installed. I built the program from the command-line using the following command: "gcc hello.cpp -ggdb -I/usr/local/include -L/usr/local/lib -lboost_thread -lc_r" The problem may be platform specific. Are you linking against a thread-safe CRT? Can you provide details regarding your operating system? Which version of the boost library do you have installed? My suspicion is that you may not be using a thread-safe CRT. Of course the boost_thread library may be broken for your specific platform (but I doubt it). Sincerely, Constantino Michailidis Balston Research LLC _______________________________________________ Boost-users mailing list Boost-users(a)lists.boost.org http://lists.boost.org/mailman/listinfo.cgi/boost-users -------------------------------------------------------- If you are not an intended recipient of this e-mail, please notify the sender, delete it and do not read, act upon, print, disclose, copy, retain or redistribute it. Click here for important additional terms relating to this e-mail. http://www.ml.com/email_terms/ --------------------------------------------------------

1 0

RE: [Boost-users] Re: MultiIndexSet invariant test failure
by Manges, Ross G 14 Apr '05

14 Apr '05

> OK, this is a different story. Yes, unfortunately. > 2. The comparison predicate you're passing is not correct (i.e. it > does not implement a strict weak ordering.) OK, this is possible. I'm using a function object. What exactly does it mean to implement strict weak ordering? I need to search the documentation again... > 3. Elements are dynamic (pointers to objects) and you did > a premature delete on one or some of them. I'm using boost::shared_ptr types, so they shouldn't get cleaned up while they are contained by the MultiIndexSet. However, I'm starting to suspect that the shared_ptrs are not behaving well. > 5. You're running out of memory and new() does not throw bad_alloc > (this can happen in MSVC++ 6.5 and, maybe, 7.0.) I'm using gcc on RHEL3; I don't believe this is happening. > 6. You haven't > selected a multithreaded version of the runtime library. I need to investigate this; what 'runtime library' are you referring to? > 7. You > guarded write access to an index but *not* to other indices of the > same container. I gaurded ~everything~ ;) > Questions: > > 1. I'd like to analyze this. I can run tests in GCC/Cygwin. Could your > test be run on this platform? If not, could you at least disclose the > code so that I can take a look at it? We're developing on the linux platform, so that's not a barrier, but unfortunately, it will be very difficult to convince my employer to allow me to disclose any code. Thank you, however, for offering to analyze it. > 2. Does your > debugging environment allow you to peek at the variables when the > assert happens? I'm using some acient version of gdb/ddd. I could try, although I'm not sure how to do that yet. > Thank *you* for reporting the problem. Sure, although it's probably something I'm doing wrong. I really do appreciate your help. This MultiIndexSet is perfect for my needs; I just need to get it working right!

2 1

RE: Boost thread problem
by C. Michailidis 14 Apr '05

14 Apr '05

> All, > > I have got a problem with the boost::thread package. > My idea was to facilitate the multiprocessor ability of my computer and > write a program which creates 10 threads which firstly write to the disk > certain amount of information and then they read it back and push back > certain objects. I am getting some memory access problems when pushing > back the objects to the global vector. Can anyone of you understand why > there is a problem? I really do not get it since the mutex is locked and > the access to the vector is exclusively guaranteed to the active thread. > > > Can you help? I tried your code on my system and it seems to work as expected. No exceptions were thrown, there was a file called 'file.txt' in the pwd, and TradeVector.size was 80000 after the run. Of course I needed to remove the include of proprietary code - it was replaced with a dummy/stub class like this one: class CFXTrade { public: CFXTrade() { }; CFXTrade( int id, string name, double amt ) { m_TradeID = id; m_Name = name; m_Amount = amt; }; int m_TradeID; string m_Name; double m_Amount; }; My system looks like this: FreeBSD medea.digitalescorts.net 5.4-PRERELEASE FreeBSD 5.4-PRERELEASE #0: Thu Mar 31 02:47:38 EST 2005 root@medea.digitalescorts.net:/usr/obj/usr/src/sys/MEDEA i386 and I have boost-1.32.0_2 libraries installed. I built the program from the command-line using the following command: "gcc hello.cpp -ggdb -I/usr/local/include -L/usr/local/lib -lboost_thread -lc_r" The problem may be platform specific. Are you linking against a thread-safe CRT? Can you provide details regarding your operating system? Which version of the boost library do you have installed? My suspicion is that you may not be using a thread-safe CRT. Of course the boost_thread library may be broken for your specific platform (but I doubt it). Sincerely, Constantino Michailidis Balston Research LLC

1 0

RE: Re: MultiIndexSet invariant test failure
by Manges, Ross G 13 Apr '05

13 Apr '05

> Date: Wed, 13 Apr 2005 06:58:01 +0000 (UTC) > From: Joaquin M Lopez Munoz <joaquin(a)tid.es> > Subject: [Boost-users] Re: MultiIndexSet invariant test failure > To: boost-users(a)lists.boost.org > Message-ID: <loom.20050413T084751-297(a)post.gmane.org> > Content-Type: text/plain; charset=utf-8 > > So, once you have mutex-guarded your code, the problem > does not show? Please confirm. The invariant test fails (asserts) on a call to 'insert()' regardless of my mutex gaurds. I have even gone as far as to wrap 'read' methods such as calls to 'size()' and 'empty()' with mutexes. > Other than that, you'll have to guard your code with > mutexes like you say you're doing now. Yes, I understand this, but it seems that the invariant assertion is happening and it may not (?) necessarily be related to a threading issue. According to my backtrace, it appears that I do not have a thread contention issue. There is only one thread that is performing a call to any of the container methods (the other threads are waiting on a mutex release), and that call is 'insert()', and it is causing the aforementioned invariant assert. The invariant assert does not happen during every test run, but it does happen frequently. What are the kinds of things that could cause the invariant assert? The documentation doesn't go into any great detail in this area. > Hope this helps. Best regards, Any help is most welcome; thanks!

2 1

uBLAS compatible to OpenGL?
by Malte Clasen 13 Apr '05

13 Apr '05

Hi, is the uBLAS library memory compatible to OpenGL vector and matrix operations? I'd like to use a single representation for both internal maths and the rendering interface of my application. OpenGL expects a pointer to a float or double C-array. For example 4x4 matrices are stored linearly as 16 values. When using std::vector<T> or bounded_array<T> as storage, is there a guarantee that m(0,0) is a reference to the first element of an array of (at least) 16 values that equal the linearized matrix elements? Or is there another way to get access to the low-level storage if it has a compatible memory layout? If there isn't, is it possible customize the uBLAS library by an additional "storage organization" or "adapted array" class? best regards, Malte

2 1

serializing abstract base class
by Alex Olivas 13 Apr '05

13 Apr '05

I'm trying to serialize an abstract base class. I used the BOOST_IS_ABSTRACT macro, but the compile fails with the following error... public/dataclasses/I3Particle.h:225: syntax error before `namespace' public/dataclasses/I3Particle.h:225: syntax error before `}' token line 225 of I3Particle is... BOOST_IS_ABSTRACT(I3Particle); Is is not as simple as adding this one line? I'm using gcc 3.2.3. Thanks, Alex.

3 2

Boost thread problem
by Sliwa, Przemyslaw (London) 13 Apr '05

13 Apr '05

All, I have got a problem with the boost::thread package. My idea was to facilitate the multiprocessor ability of my computer and write a program which creates 10 threads which firstly write to the disk certain amount of information and then they read it back and push back certain objects. I am getting some memory access problems when pushing back the objects to the global vector. Can anyone of you understand why there is a problem? I really do not get it since the mutex is locked and the access to the vector is exclusively guaranteed to the active thread. Can you help? Thank you in advance Pshemek #include "FXTrade.h" #include <string> #include <fstream> #include <vector> #include <iostream> #include <cmath> #include <exception> #include <boost/random.hpp> #include <boost/thread/thread.hpp> #include <boost/thread/mutex.hpp> #include <boost/bind.hpp> using namespace std; using namespace boost; namespace write_namespace { mutex file_mutex; ofstream out; const size_t dimension(10000); const size_t size(10); const unsigned int number_of_threads(8); } vector<CFXTrade> TradeVector; namespace read_namespace { mutex in_file_mutex; ifstream in; } void write_to_disk(const vector<CFXTrade>& nVector) { using namespace write_namespace; mutex::scoped_lock lock(file_mutex); for(vector<CFXTrade>::const_iterator i_ptr=nVector.begin(); i_ptr!=nVector.end(); ++i_ptr) { out << i_ptr->m_TradeID << ";" << i_ptr->m_Name << ";" << i_ptr->m_Amount << endl; } } auto_ptr<CFXTrade> getTradeFromString(string& c_tmp) { int i_TradeID; string c_Name, c_sub; string::size_type pos; double d_Amount(0.0); pos = c_tmp.find(";"); c_sub = c_tmp.substr(0, pos); i_TradeID = atoi(c_sub.c_str()); c_sub = c_tmp.substr(++pos, 3); c_Name = c_sub; c_sub = c_tmp.substr(pos+4, 3); d_Amount = atof(c_sub.c_str()); d_Amount = 100.; return auto_ptr<CFXTrade>(new CFXTrade(i_TradeID, c_Name, d_Amount)); } void read_from_disk(vector<CFXTrade>& nTradeVector) { using namespace write_namespace; using namespace read_namespace; string c_tmp; try { for(size_t i=0; i<dimension; ++i) { mutex::scoped_lock lock(in_file_mutex); getline(in, c_tmp); TradeVector.push_back(*getTradeFromString(c_tmp) ); } } catch(exception& e) { cout << "A standard exception caught from " << __LINE__ << endl; cout << e.what() << endl; } } bool generate_file() { using namespace write_namespace; vector< thread* > ThreadVector(number_of_threads); mt19937 mt2; random_number_generator<mt19937> std_rng(mt2); int TradeIDVector[] = {1, 2, 3, 4, 5, 6, 7, 8, 9, 10}; char* NameVector[] = {"ABC", "BCD", "CDE", "DEF", "EFG", "FGH", "GHI", "HIJ", "IJK", "JKL"}; double AmountVector[] = {10., 20., 30., 40., 50., 60., 70., 80., 90., 11.}; vector< vector<CFXTrade> > TradeVector(number_of_threads); for(int i=0; i<number_of_threads; ++i) { for(int j=0; j<dimension; ++j) { TradeVector[i].push_back(CFXTrade(TradeIDVector[std_rng(size)], string(NameVector[std_rng(size)]), AmountVector[std_rng(size)])); } } out.open("file.txt"); for(int i=0; i<number_of_threads; ++i) { ThreadVector[i] = new thread(bind(&write_to_disk, TradeVector[i])); } for(int i=0; i<number_of_threads; ++i) { ThreadVector[i]->join(); } out.close(); for(int i=0; i<number_of_threads; ++i) { delete ThreadVector[i]; } return true; } bool read_trades() { using namespace write_namespace; using namespace read_namespace; vector<thread*> ReadThreadVector(number_of_threads); in.open("file.txt"); try { for(int i=0; i<number_of_threads; ++i) { ReadThreadVector[i] = new thread(bind(&read_from_disk, TradeVector)); } for(int i=0; i<number_of_threads; ++i) { ReadThreadVector[i]->join(); } in.close(); for(int i=0; i<number_of_threads; ++i) { delete ReadThreadVector[i]; } } catch(exception& e) { cout << "A standard exception caught from " << __LINE__ << endl; cout << e.what() << endl; return false; } return true; } int main() { using namespace read_namespace; try { generate_file(); read_trades(); } catch(exception& e) { cout << e.what() << endl; exit(EXIT_FAILURE); } catch(...) { cout << "Unknown exception occured" << endl; } cout << TradeVector.size() << endl; } -------------------------------------------------------- If you are not an intended recipient of this e-mail, please notify the sender, delete it and do not read, act upon, print, disclose, copy, retain or redistribute it. Click here for important additional terms relating to this e-mail. http://www.ml.com/email_terms/ --------------------------------------------------------

1 0

boost::variant and operator<<
by Matt Hortman 13 Apr '05

13 Apr '05

Hello, I feel almost silly asking this but: I can only get the code listed below to compile if I remove the comments on the namespace statements, placing my custom operator<< in the boost::detail::variant namespace. Otherwise I get the error c:\Boost\include\boost-1_32\boost\variant\detail\variant_io.hpp(64) : error C2679: binary '<<' : no operator found which takes a right-hand operand of type 'const T1' (or there is no acceptable conversion) from VC++ 7.1 and a similar error from both gcc 3.3 and 3.4. I'm using boost 1.32 on both WinXP and FreeBSD. Looking at the code in variant_io.hpp, I see no reason why it should not work without the namespace statements. Is there something fundamental about namespaces that I don't understand? Thanks, -Matt #include <vector> #include <string> #include <iostream> #include <boost/variant.hpp> typedef std::vector<std::string> strlist; typedef boost::variant<int, strlist> ConfigItem; //namespace boost { namespace detail { namespace variant { inline std::ostream& operator<< (std::ostream& os, const strlist& slist) { strlist::const_iterator it; for (it=slist.begin(); it!=slist.end(); ++it) os << *it << ", "; return os; } //}}} //using namespace boost::detail::variant; void main (void) { strlist sl; std::cout << sl << std::endl; //compiles fine ConfigItem v; std::cout << v << std::endl; // error no operator found }

3 2