July 2009 - Boost-users - lists.preview.boost.org

Tokenizer design question
by Polder, Matthew J 14 Jul '09

14 Jul '09

The Tokenizer library has a char_separator with the option to keep delimiters, drop delimiters, and keep or drop empty tokens. However, with escaped_list_separator, the only behavior is to keep empty tokens. While this is the obvious behavior for parsing csv and similar files, it would be nice to have the ability to also drop empty tokens when constructing an escaped_list_separator. I have a command line parser that either reads its arguments from the command line itself or a text file supplied on the command line. In the file I'm passing in formats for the Date Time library I/O routines, and the formats have spaces that I'm escaping so the format will be a single token, which Tokenizer does find. But I sometimes use multiple tabs to separate my fields so it will look pretty in a text editor, and escaped_list_separator is keeping these. The solution for now is to have a switch in my command line parser for which separator I want to use. thanks, matthew

2 1

A linking error on switch_statement. Can anyone with kindness help me?
by fmingu 14 Jul '09

14 Jul '09

I am using Dev-C++ with boost 1.34.1,the program is the following: ....................... typedef std::map<int, int> IIPrimemap; IIPrimemap primemapvec; # define BetaKGre2 1 # define EqualToOne 2 #define ProductOfKPrime 3 ......................... (switch_statement(_1, case_statement<BetaKGre2> (var(result)=0), case_statement<EqualToOne> (var(result)=1), case_statement<ProductOfKPrime> (var(result)=bind(static_cast<long double (*)(long double,int)>(std::pow), constant(-1.0),bind(&IIPrimemap::size,var(primemapvec)))), default_statement ( var(result)=234) ))(switchvalue); the complier told me no errors.But the linker told me that: nstantiated from `Arg::sig<boost::tuples::tuple<A&, B&, C&, Env&, boost::tuples::null_type, boost::tuples::null_type, boost......... instantiated from `RET boost::lambda::lambda_functor_base<boost::lambda::s................ nstantiated from `T::sig<boost::tuples::tuple<A&, boost::tuples::null_type, boost::tuples::null_type, bo........... instantiated from here..........// indicating the line: ))(switchvalue); And after the second compling and linking , the program passed and ran. I do not know why. And this was caused after I added the line: (var(result)=bind(static_cast<long double (*)(long double,int)>(std::pow), constant(-1.0),bind(&IIPrimemap::size,var(primemapvec)))), otherwise the program do not have such problems. How can I solve it? Can anyone with kindness help me?

2 1

Boost tokenizer does not work, and shows up invalid read of size on valgrind
by Avi Bahra 14 Jul '09

14 Jul '09

Using latest boost version 39 on gcc 4.2.1 on SUSE linux I am trying to load a file, and then split the lines into a vector of strings. However when this is run, it showed that the last string was corrupt. When I ran this with valgrind, the very first error shows an invalid read of size 1, in guts of boost::char_separator. ==19963== Invalid read of size 1 ==19963== at 0x8056870: bool boost::char_separator<char, std::char_traits<char> >::operator()<__gnu_cxx::__normal_iterator<char const*, std::string>, std::string>(__gnu_cxx::__normal_iterator<char const*, std::string>&, __gnu_cxx::__normal_iterator<char const*, std::string>, std::string&) (token_functions.hpp:430) ==19963== by 0x8056E3E: boost::token_iterator<boost::char_separator<char, std::char_traits<char> >, __gnu_cxx::__normal_iterator<char const*, std::string>, std::string>::initialize() (token_iterator.hpp:70) ==19963== by 0x8056EA6: boost::token_iterator<boost::char_separator<char, std::char_traits<char> >, __gnu_cxx::__normal_iterator<char const*, std::string>, std::string>::token_iterator(boost::char_separator<char, std::char_traits<char> >, __gnu_cxx::__normal_iterator<char const*, std::string>, __gnu_cxx::__normal_iterator<char const*, std::string>) (token_iterator.hpp:77) ==19963== by 0x8056FAA: boost::tokenizer<boost::char_separator<char, std::char_traits<char> >, __gnu_cxx::__normal_iterator<char const*, std::string>, std::string>::begin() const (tokenizer.hpp:86) here is program: BOOST_AUTO_TEST_CASE( test_log_append ) { string logFile = "test/logfile.txt"; // Load the log file into a vector, of strings, and test content ifstream ifs(logFile.c_str()); BOOST_REQUIRE_MESSAGE(ifs, "Could not open log file\n"); stringstream ss; ss << ifs.rdbuf(); // Read the whole file into a string char_separator<char> sep("\n"); // Split the file content unix=\n pc =\n\r typedef boost::tokenizer<boost::char_separator<char> > tokenizer; tokenizer tokens(ss.str(), sep); // <<<<<<<< valgrind barfs here std::vector<std::string> lines; lines.reserve(9); std::copy(tokens.begin(), tokens.end(), back_inserter(lines)); // <<<<<<<< valgrind barfs here for(int i = 0; i < lines.size(); i++) { cerr << "'" << lines[i] << "'\n"; } } the input in the logfile.txt is of the form: MSG:[16:36:09 14.7.2009] First Message LOG:[16:36:09 14.7.2009] LOG WAR:[16:36:09 14.7.2009] ERROR ERR:[16:36:09 14.7.2009] WARNING DBG:[16:36:09 14.7.2009] DEBUG OTH:[16:36:09 14.7.2009] OTHER OTH:[16:36:09 14.7.2009] OTHER2 MSG:[16:36:09 14.7.2009] Last Message The output is of the form: 'MSG:[16:36:09 14.7.2009] First Message' 'LOG:[16:36:09 14.7.2009] LOG' 'WAR:[16:36:09 14.7.2009] ERROR' 'ERR:[16:36:09 14.7.2009] WARNING' 'DBG:[16:36:09 14.7.2009] DEBUG' 'OTH:[16:36:09 14.7.2009] OTHER' 'OTH:[16:36:09 14.7.2009] OTHER2' '�:[16:36:09 14.7.2009] Last Message' Notice that the last string is corrupt. Is the tokenizer known to be buggy in boost 1.39, or am I doing it all wrong ? Best regards, Ta, Avi

2 1

filesystem filename/regexp pattern matching, fnmatch
by Mark Lohry 14 Jul '09

14 Jul '09

I'm porting some linux software to Windows/MSYS, and I've run into a number of linux-specific file system routines that I was hoping boost::filesystem would be able to render portable, specifically "glob" and "fnmatch". As for glob, I came across this thread: http://article.gmane.org/gmane.comp.lib.boost.user/5198/match=glob but it appears to not have been implemented into boost, and the link to source code is long dead. Am I going to need to just re-write all glob-specific code into something boost::filesystem compatible? As for fnmatch(), which matches regular expressions in strings (say, finding all files in a directory which contain a numeral), does this not exist in the filesystem library, or am I overlooking something? Sorry for the novice nature of the question, but my Boost experience is limited to numerics. -ML

1 0

regex traits for glib::ustring
by Simon Wibberley 14 Jul '09

14 Jul '09

Hi there, I'm attempting to build an application which requires multibyte character support, for which I'm using the Glib::ustring, and needs also to use regex. From what I understand you need to tell boost regex exactly how to iterate over your strings. I've got as far as: typedef boost::match_results<Glib::ustring::iterator, std::allocator<gunichar> > umatch; typedef boost::reg_expression<Glib::ustring, boost::regex_traits<Glib::ustring> > uregex; //making the regex_match<ustring::iterator, std::allocator<gunichar>, ustring, regex_traits<ustring> >(str.begin(), str.end(), m, re); //where str is a ustring, re is a uregex and m is a umatch as defined above However I get the feeling I'm completely missing a trick, ie that's it's a lot more complex than this... I'm using boost 1.39 and glibmm 2.4. Could anyone provide any guidance on this, whether it's even possible without writing a lot of code, or possible alternatives? Many thanks, Simon

2 1

[test] Typo in documentation?
by Christian Henning 14 Jul '09

14 Jul '09

Hi there, on the page: http://www.boost.org/doc/libs/1_39_0/libs/test/doc/html/utf/user-guide/runt… the command line argument for detecting memory leaks is "build_info". Shouldn't that be "detect_memory_leak"? Regards, Christian

1 0

Re: [Boost-users] Date Time I/O formatting question
by Polder, Matthew J 14 Jul '09

14 Jul '09

Commander Pirx wrote: ------------------------------------------ Would be interesting whats your format string looks like. Not all format flags work with all date_time classes. %z (Time zone) for instance works only with local_date_time. ------------------------------------------ It seems that no matter what formatting I try to use, it doesn't work. Example: TimeConverter sampleFile.txt %d:%m:%Y_%H:%M:%S %m:%d:%Y_%H:%M:%S sampleFile.txt: 05:01:2007_23:08:02 06:01:2007_23:09:04 expected output: (month and day positions switched) 2007-Jan-05 23:08:02 01:05:2007_23:08:02 2007-Jan-06 23:09:04 01:06:2007_23:09:04 output: 2007-Jan-05 23:08:02 2007-Jan-05 23:08:02 2007-Jan-06 23:09:04 2007-Jan-06 23:09:04 Code is below #include <fstream> #include <functional> #include <iostream> #include <sstream> #include "boost/date_time/gregorian/gregorian.hpp" #include "boost/date_time/posix_time/posix_time.hpp" #include "boost/date_time/local_time/local_time.hpp" using namespace std; int main(int argc, char** argv) { stringstream ss; ifstream in( argv[1] ); string inputFormat = argv[2]; string outputFormat = argv[3]; boost::local_time::local_time_facet* output_facet = new boost::local_time::local_time_facet(); boost::local_time::local_time_input_facet* input_facet = new boost::local_time::local_time_input_facet(); ss.imbue(locale(locale::classic(), output_facet)); ss.imbue(locale(ss.getloc(), input_facet)); output_facet->format( outputFormat.c_str() ); input_facet->format( inputFormat.c_str() ); while ( !in.eof() ) { string s; getline( in, s ); ss<<s; boost::posix_time::ptime p; ss>>p; string s1 = boost::posix_time::to_simple_string(p); cout<<s1<<"\n"; ss.str(""); ss<<p; string s2 = ss.str(); cout<<s2<<"\n"; std::ws(in); ss.str(""); } }

1 0

BOOST::Test floating_point_comparison.hpp:224: warning: ‘boost::test_tools::<unnamed>::check_is_close’ defined but not used
by Avi Bahra 14 Jul '09

14 Jul '09

I am using boost 1.39 on gcc , I have many tests, but keep getting the warning above. Is there any way to get rid of this compiler warning ? Any help appreciated. Best regards, Ta, Avi

1 0

Build error
by OvermindDL1 14 Jul '09

14 Jul '09

I am attempting to build boost, first I set some environment vars (some may not be useful anymore, but I still set them): set BZIP2_SOURCE=G:\SDKs\boost\bzip2-1.0.4 set ZLIB_SOURCE=G:\SDKs\boost\zlib-1.2.3 set HAVE_ICU=1 set ICU_PATH=R:\SDKs\icu\icu set EXPAT_INCLUDE=G:\SDKs\expat\expat_cvs\expat\lib set EXPAT_LIBPATH=G:\SDKs\expat\expat_cvs\expat\win32\bin\release I then call: bootstrap.bat bjam --build-type=complete --toolset=msvc-8.0 --without-mpi --without-python --prefix=R:/SDKs/boost/built_head --build-dir=R:/SDKs/boost/build_head define=_CRT_NONSTDC_NO_DEPRECATE define=_CRT_SECURE_NO_DEPRECATE define=_SECURE_SCL=0 define=_SCL_SECURE_NO_DEPRECATE define=_HAS_ITERATOR_DEBUGGING=0 install>..\build-HEAD.log And my build-HEAD.log contains: Note: Building Boost.Regex with Unicode/ICU support enabled Using ICU in R:\SDKs\icu\icu/include warning: Graph library does not contain MPI-based parallel components. note: to enable them, add "using mpi ;" to your user_config.jam error: link=shared together with runtime-link=static is not allowed error: such property combination is either impossible error: or too dangerious to be of any use I have never seen those last three errors, and other then the bootstrap my commands have not changed in many many versions. What is causing them?

2 3

Boost Regex: Use boost::uint32_t as character type.
by Etienne Philip Pretorius 14 Jul '09

14 Jul '09

Hello List, I just like to know, if you can use a std::vector<boost::uint32_t> as a source to match regular expressions against it. Etienne

2 4