Re: [Boost-users] <boost::wave> Memory issue when using wave to run a file parser service

I run into a memory issue when uses boost wave library. My application runs as a service that parses c files. In each individual run, it calls in the wave library through a sub-function call to process one file. I noticed that some memory is not released from wave after each run. The consequence is that the memory usage of the application keeps increasing till the system breaks down.

I could reproduce the problem with a simple modification of the quick_start sample code shipped with the boost wave library. (attached below).

My system is win64 and I see the problem with both 1.49 and 1.55. (built with b2 toolset=msvc-11.0 address-model=64 --build-type=complete)

Something I found during the investigation: * The problem is gone if I build the application with link<static>. (Unfortunately, this is not an option for me) * From the memory usage pattern, it seems that there is a memory pool in place that should have been released after each turn. When repeatedly passing a 4MB file, the memory usage (taken from task maneger) is: 99,700K, 198,404K, 395,404K, 395408K, and 789,396K. When running with static link, the memory usage after each run is: 1800K, 2480K, 2480K, .... The maximum usage is 8300K. Is there any workaround with the problem?

Thanks for reporting this. I'll have a look asap. Regards Hartmut --------------- http://boost-spirit.com http://stellar.cct.lsu.edu

----

Thanks, Yong

------------------------------------- begin of the code ------------------ ---------------------------------

#define BOOST_WAVE_SERIALIZATION 0 // enable serialization #define BOOST_WAVE_BINARY_SERIALIZATION 0 // use binary archives #define BOOST_WAVE_XML_SERIALIZATION 1 // use XML archives ////////////////////////////////////////////////////////////////////////// ///// ////////////////////////////////////////////////////////////////////////// ///// // Include Wave itself #include ////////////////////////////////////////////////////////////////////////// ///// // Include the lexer stuff #include // token class #include // lexer class // sub-function that calls into wave int do_it(std::string filename) { boost::wave::util::file_position_type current_position; std::ifstream instream(filename); std::string instring; instream.unsetf(std::ios::skipws); instring = std::string(std::istreambuf_iterator<char>(instream.rdbuf()), std::istreambuf_iterator<char>()); typedef boost::wave::cpplexer::lex_token<> token_type; typedef boost::wave::cpplexer::lex_iterator lex_iterator_type; typedef boost::wave::context context_type; typedef boost::wave::cpplexer::lex_token<> TokenType; typedef boost::wave::cpplexer::lex_iterator<TokenType> LexIterator; LexIterator aIter(instring.begin(), instring.end(), TokenType::position_type( filename.c_str()), boost::wave::language_support( boost::wave::support_cpp | boost::wave::support_option_long_long ) ); LexIterator endIter; for (; aIter != endIter; ) { (*aIter).get_value(); ++aIter; } return 0; } int main(int argc, char *argv[]) { int aNum = 1; for (;;) { do_it("c:\\memleak.txt"); std::cout << "Process the file again (0/1)? \n"; scanf("%d", &aNum); if (aNum == 0) { printf("\nYou entered zero so we stop!\n"); return 0; } } return 0; }