February 2010 - Boost-users - lists.preview.boost.org

[proto] transform for a templated proto::terminal type
by Manjunath Kudlur 06 Feb '10

06 Feb '10

Hello, I have some proto::terminal's as follows : template<typename T> struct Var { unsigned int id; }; typedef proto::terminal<Var<int> >::type int32_; typedef proto::terminal<Var<float> >::type float_; typedef proto::terminal<Var<double> >::type double_; I can write a callable transform, with templated operator() functions to handle each Var<> type as follows : struct _var_tostr : proto::callable { typedef std::string result_type; template<typename T> result_type operator()(variables::Var<T> const &v) const { std::stringstream sout; sout << "S" << v.id; return sout.str(); } template<typename T> result_type operator()(variables::VecVar<T> const &v) const { std::stringstream sout; sout << "V" << v.id; return sout.str(); } }; To activate the transform, I end up writing one rule for each Var<> type as follows : struct var_tostr : proto::or_< proto::when<proto::terminal<variables::Var<int> >, _var_tostr(proto::_value)> ,proto::when<proto::terminal<variables::Var<float> >, _var_tostr(proto::_value)> ,proto::when<proto::terminal<variables::Var<double> >, _var_tostr(proto::_value)> > { }; This gets cumbersome when I have more Var<> types. Is there a way to not list each Var<> type in the grammar? That is, I want to match any Var<T> and just call the _var_tostr(proto::_value) transform. How can I accomplish this? Thanks, Manjunath

2 1

[Serialization] Speedding up client-server communication
by Ruediger Berlich 06 Feb '10

06 Feb '10

Hi there, I am in the process of speeding up communication between a server and its clients. Communication involves serialized class data. Messages can be as large as 100 kilobytes. I have done some measurements which have shown that, in a cluster with Gigabit networking, most overhead of the parallelisation seems to come from the Broker infrastructure and the process of (de-)serialization. Network latency and/or bandwidth seems to play only a minor role in this environment. Hence, apart from optimizing my broker, I'm looking for ways to optimize the serialization process, as used in my application. As messages are discarded as soon as they reach the recipient, versions of serialized data do not play an important role. Apart from decreasing the frequency and size of data exchanges and using binary transfers in a homogeneous environment, another way of speeding up the application seems to be the "Class Information" section discussed e.g. in http://www.boost.org/doc/libs/1_41_0/libs/serialization/doc/special.html . Also, BOOST_IS_BITWISE_SERIALIZABLE(my_class) might help. One question I have about this is whether a std::vector<POD> would be accessible to this optimization ? I use quite a few of them. Do you have further suggestions for ways of influencing the Boost.Serialization library ? Thanks and Best Regards, Ruediger

3 5

[Interprocess] How to make a clean print using a managed file mapping?
by Alexander Maslowski 06 Feb '10

06 Feb '10

Interprocess newbie question. I’m trying to use interprocess to print a data file to possibly improve the speed of my code. I’ve attached a code sample to the bottom. The code compiles with msn cl compiler, but segfaults with gcc (4.3.2). Two questions: Is it possible to produce a file that is not junked up with binary at the beginning and end? Any ideas why it seg-faults with gcc? Thanks in advance. --Alex #include <boost/interprocess/managed_mapped_file.hpp> #include <boost/interprocess/streams/bufferstream.hpp> #include <boost/filesystem/path.hpp> #include <boost/filesystem/operations.hpp> #include <exception> #include <iostream> #include <vector> int main( int /*argc*/, char** /*argv*/ ) { using namespace std; using namespace boost; //Initialize the data to be printed size_t dataSetSize = 10; vector<double> coordinate(dataSetSize); vector<double> magnitude (dataSetSize); for( size_t i = 0; i < dataSetSize; ++i){ coordinate[i] = static_cast<double>(i); magnitude [i] = static_cast<double>(2*i) + static_cast<double>(2*i)/1e10; } vector<string> banner(2); banner[0]="this is my data file banner\n"; banner[1]="file end"; string fileName("fileMapperTest.out"); try{ using namespace filesystem; using namespace interprocess; //Remove the file if it already exists path fileNamePath(fileName); if( exists(fileNamePath) ) remove(fileNamePath); size_t dataWidth = 20; size_t bufferSize = banner[0].size()+2*dataSetSize*(dataWidth+3)+banner[1].size(); //map the file managed_mapped_file tstFileMap( create_only, fileName.c_str(), 2*bufferSize); cout << "file was mapped" << endl; //create a buffer char *aString = tstFileMap.construct<char>("aString") [bufferSize](); cout << "the buffer was constructed" << endl; //stream into the buffer bufferstream testFileBuff(aString, bufferSize, ios_base::out); testFileBuff.precision(dataWidth-5); cout.precision(dataWidth-5); cout << "the buffer stream was constructed" << endl; //Start printing testFileBuff << banner[0] << "\n"; for( size_t i = 0; i < dataSetSize; ++i){ testFileBuff << coordinate[i] << " " << magnitude [i] << "\n"; cout << coordinate[i] << " " << magnitude [i] << "\n"; } testFileBuff << banner[1] << endl; tstFileMap.destroy_ptr(aString); } catch(interprocess::interprocess_exception& e) { cout << "An interprocess exception was thrown ..." << endl; cout << e.what() << endl; } catch(...) { cout << "An unknown exception was thrown ..." << endl; } return 0; } My file output: [binary junk]... 0 0 1 2.0000000002 2 4.0000000004 3 6.0000000006 4 8.0000000008 5 10.000000001 6 12.0000000012 7 14.0000000014 8 16.0000000016 9 18.0000000018 file end [more binary junk]

1 0

lambda control structures
by Archie14 06 Feb '10

06 Feb '10

I have following code: bool a = true; std::for_each (_params.begin(), _params.end(), boost::lambda::if_(! boost::lambda::bind (&inputParams::getParamValue, this, boost::lambda::_1, boost::ref(_fddata))) [a = false]); if (! a) ... What I am trying to achieve is to stop for_each if "getParamValue" returns false. However I can't find a way to use "return" in the second part of boost::lambda::if_, so I am using a dummy var "a" to check it after the loop completes. My question is: what is the right way to use lambda control structure in this case?

3 3

Error using boost::thread
by Daniel Veneros 05 Feb '10

05 Feb '10

Hi, I was trying to work with boost::thread but I receive an error just by including the header: #include <boost/thread.hpp> #include <boost/date_time.hpp> Error: from main.cpp:17: In file included from /usr/include/boost/algorithm/string/find_format.hpp:22, /usr/include/boost/algorithm/string/concept.hpp:38: error: expected `;' before '<' token /usr/include/boost/algorithm/string/concept.hpp:46: error: `range' does not name a type /usr/include/boost/algorithm/string/concept.hpp: In member function `void boost::algorithm::FinderConcept<FinderT, IteratorT>::constraints()': /usr/include/boost/algorithm/string/concept.hpp:43: error: `r' undeclared (first use this function) /usr/include/boost/algorithm/string/concept.hpp:43: error: (Each undeclared identifier is reported only once for each function it appears in.) /usr/include/boost/algorithm/string/concept.hpp: In member function `void boost::algorithm::FormatterConcept<FormatterT, FinderT, IteratorT>::constraints()': /usr/include/boost/algorithm/string/concept.hpp:68: error: `begin' is not a member of `boost' /usr/include/boost/algorithm/string/concept.hpp:69: error: `end' is not a member of `boost' Any idea how to solve this or what is causing this errors? Thanks Dann

2 1

newbie question - memory leak while deserializing an object
by Moldovanyi Orban 05 Feb '10

05 Feb '10

Hi there! :) I've recently become familiar with boost, therefore I'm quite newbie and maybe the answer to my question is trivial. Below there is a demonstrative code fragment from a test client application where a server app periodically sends serialized objects to the client. The client reads the socket, re-constructs the serialized object from the stream using boost::archive::text_iarchive and makes some operation on it. Code is very simple and it works fine, however the memory is leaking. I think there is a problem with the usage of text_iarchive because if I comment the appropriate line out, the leaking stops. Of course the incomingTransferObject (and everything else in the progbam) are destructed properly, they does nothing with the leak. Here comes the code fragment: char buffer[8192]; // class memeber, it contains data received on the socket ... void Client::handleReadMessage(const boost::system::error_code& error, std::size_t bytes_transferred) { // blah-blah ... TransferObject* incomingTransferObject = new TransferObject(); std::istringstream archive_stream(buffer); boost::archive::text_iarchive archive(archive_stream); *archive >> incomingTransferObject;* //... // do something with the object //... delete incomingTransferObject; tcpSocket->write_some(boost::asio::buffer("1")); // ack to the server boost::asio::async_read( *tcpSocket, boost::asio::buffer(sizeBuffer), boost::asio::transfer_at_least(1), boost::bind( &Client::handleReadHeader, this, boost::asio::placeholders::error, boost::asio::placeholders::bytes_transferred) ); } The TransferObject is a temporary class just to test serializing more complex objects with stl containers. It looks like this: class TransferObject { friend class boost::serialization::access; template<typename Archive> void serialize(Archive & ar, const unsigned int version) { ar & strvector; ar & strmap; ar & complexSet; } private: vector<string>* strvector; map<int,string>* strmap; set<HelperObject*>* complexSet; ... } class HelperObject { friend class boost::serialization::access; template<typename Archive> void serialize(Archive & ar, const unsigned int version) { ar & b; ar & f; } private: bool b; float f; ... } >From my point of view it seems that the text_iarchive makes some memory allocations during its operation but their references are lost after the deserialization which cause the leak, anyway I'm almost absolutely sure that I'm using this technique badly and there is a simple solution for this problem. Thanks for reading this and I hope an expert can answer. :] Laszlo

1 0

Switching from CreateThread to boost::thread
by Kurt D. Knudsen 05 Feb '10

05 Feb '10

Hi all, Currently, I'm working on a Windows C++ program that was developed by a third party and I'm just working out the kinks and adding new features. I'm not 100% familar with threads, but I think I have the general idea down. The project is a client/server backup solution. The problem I am running into is this after the code pasted below: void CTransferServer::startCommandThread() { int iCommandPort = _wtoi(configHandler->getTextValue(L"TransferCommandPort")); SOCKET socCommand = CSocHandler::getServerSocket(iCommandPort); if (socCommand == INVALID_SOCKET) { //cout << "\nCould not create transfer command server socket." << WSAGetLastError(); return; } if (listen(socCommand, SOMAXCONN)) { //cout << "\nCould not listen on transfer command port." << WSAGetLastError(); return; } while (blStarted) { SOCKET sa = accept(socCommand, 0, 0); // block for connection request if (sa ==INVALID_SOCKET) { break; } void **params = (void **) malloc(sizeof(void*)*2); SOCKET *s = new SOCKET; *s = sa; params[0] = (void*)this; params[1] = (void*)s; DWORD dwGenericThread; //unsigned int iGenericThread; HANDLE hWorkerThread = CreateThread(NULL, 0, transferCommandWorkerThread, params, 0, &dwGenericThread); //HANDLE hWorkerThread = (HANDLE)_beginthreadex(NULL, 0, transferCommandWorkerThread, params, 0, &iGenericThread); wcout << "Transfer Server worker command handle: " << hWorkerThread << endl; //WaitForSingleObject(hWorkerThread, INFINITE); //CloseHandle(hWorkerThread); } } You can see the while-loop that creates a socket that accepts a connection, it then passes this socket off as a parameter used in the thread creation. Here's the code for the transferCommandWorkerThread: DWORD WINAPI transferCommandWorkerThread(LPVOID param) { void **params = (void **)param; CTransferServer *transferServer = (CTransferServer*)params[0]; SOCKET *commandSocket = (SOCKET*)params[1]; transferServer->handleCommandConnection(*commandSocket); delete commandSocket; free((void*)params); return 0; } So this workerThread passes the socket off to another function that handles the command connection and then deletes/frees the data used. All that aside, here's the issue. This while-loop generates hundreds of thousands of handles in a matter of days. This causes all kinds of problems on the server (there's a similar issue with clients using identical functions) and either the server needs to be rebooted or the program shut down. I tried to use WaitForSingleObject() and then CloseHandle() but it seems to close the handle prematurely. This causes the client/server to not communicate and interrupt any data transfer. I am wondering if using boost::thread will make handling the destruction of the threads easier. I've been researching it but I can't find any solid information on whether or not CloseHandle(), or something similar, needs to be run. I'm just trying to see if using Boost is the right direction to go. Thanks, Kurt

4 4

JSON and ptree questions
by Gustaf Johansson 05 Feb '10

05 Feb '10

Hi, I've been using the JSON writer with the property_tree. I find it difficult to write lists, since ptree::add does not allow empty paths, and that is required for the json writer to write lists. I have experimentally commented out two asserts, namely line 851, ptree_implementation.hpp: "assert(!p.empty() && "Empty path not allowed for put_child.");" line 200, string_path.hpp: "assert(!empty() && "Reducing empty path");" This is the only way that I've found to correctly write JSON lists in debug mode. Is there any way to do this without the trouble of commenting out the lines above? If not, would it not be a good thing to consider some other way to handle lists in the lib? / Gustaf

1 0

Ref: large data sets and BOOST_FOREACH / boost:bind don't mix ?
by Avi Bahra 05 Feb '10

05 Feb '10

You guys were right, I forgot that it was debug release when run in release mode, I get: Time: BOOST_FOREACH(Fred& fred, vec) { fred.inc(); } 0.28 Time: std::for_each(vec.begin(),vec.end(),boost::bind(&Fred::inc,_1) ); 0.56 Time: for (std::vector<Fred>::iterator i = vec.begin(); i < theEnd ; i++) { (*i).inc(); } 0.27 Time: std::for_each(vec.begin();vec.end(),std::mem_fun_ref(&Fred::inc)) 0.6 Time: for (size_t i = 0; i < theSize ; i++) { vec[i].inc(); } 0.27 Best regards, Ta, Avi

1 0

Re: [Boost-users] large data sets and BOOST_FOREACH / boost:bind don'tmix ?
by pete＠pcbartlett.com 05 Feb '10

05 Feb '10

>[Comparison of loops] >I get following times when run on suse linux with gcc 4.2.1/ boost 1.39 I'm sorry - I totally failed to read your post properly where you state you are using GCC. My apologies. In this case, I, like Roman, suspect the optimization settings are not right.

1 0