- Boost-users - lists.preview.boost.org

Re: [Boost-users] Tokenizer Question
by Victor A. Wagner Jr. 12 Jun '05

12 Jun '05

At 07:33 2005-06-12, Tom Browder wrote: >Victor, my humble tokenizer: > > string input_to_be_tokenized; > istringstream ss; > string s; > deque<string> tokens; > while (input_to_be_tokenized >> ss) > tokens.push_back(ss); > >I made 3 test programs: > string inp; > for (int i = 0; i < 10000000; ++i) > inp += " a"; > // generate a container of tokens from inp with one of three methods: > // my method (see above) > // Victor's method > // Boost, using a separator of (" \n\t") > > // if desired, loop over all the tokens using operator[] for the two > deques, and the iterator for Boost's container > >Then I compiled them (gcc 3.4.2, Fedora Core 3, i386): > > g++ -pg progN progN.cc > >Ran them without the final loop and saved 'gmon.out' as a unique name. > >Ran them again with the final loop and saved 'gmon.out' as a unique name. > >Ran all six gmon's and saved the outputs to unique files: > > gprof progN gmonX > X.prof > >The accumulated times (sec) are surprising: > > Boost Victor's Mine > ===== ===== ==== >no loop 1.50 38.84 20.13 >loop 131.91 38.89 23.91 it's difficult to fathom a hand written loop beating the internal constructor for deque > >Granted, I didn't do the tests multiple times, but it seems to me that the >Boost tokenizer is great if you don't need to iterate through it, but it >is the pits if you do. > >-Tom > >I'll send you my code and results if interested. > >-Tom > > prog > >Ran then a >Ran them with gprof > > gprof progN > >Ran > generate a string with 10,000,000 tokens (" a")_: " a a a ....a" and > timed your tokenizer against mine 10 times. Mine beat yours by 2 to 3 > seconds every time. > >The I used the Boost tokenizer and the timings went WAY down. > >So I think the benefits og the Boost tokenizer are well worth it, even for >trivial tokenizing. > >-Tom > > > >my tokenizer > > >---------- >From: boost-users-bounces(a)lists.boost.org >[mailto:boost-users-bounces@lists.boost.org] On Behalf Of Victor A. Wagner Jr. >Sent: Sunday, June 12, 2005 1:20 AM >To: boost-users(a)lists.boost.org >Subject: Re: [Boost-users] Tokenizer Question > >At 19:19 2005-06-11, you wrote: >> > for tokenizing on whitespace, simple stream input (>>) to a >> > std::string suffices. >> >>My own tokenizer does just that--and puts the tokens into a deque. >> >> > IMO, it's hardly worth troubling yourself with a tokenizer >> > for whitespace. >> >>Well, not really. When parsing line-oriented output and semi-known >>structured lines it's handy to be able to sometimes work with a line's >>tokens as if they were in a vector or deque. > > string yourline; > istringstream is( yourline ); > deque < string > yourvec(( istream_iterator < std :: string >( is > )), istream_iterator <std :: string >()); > >voila, a deque > >it would be interesting to profile that against the hypothetical indexable >tokenizer. > > >>In fact, I was going to add a suggestion that the tokenizer also have the [] >>operator so that the individual tokens could be addressed as tok[1], etc. >> >>-Tom >> >>_______________________________________________ >>Boost-users mailing list >>Boost-users(a)lists.boost.org >>http://lists.boost.org/mailman/listinfo.cgi/boost-users > >Victor A. Wagner Jr. http://rudbek.com >The five most dangerous words in the English language: > "There oughta be a law" > >_______________________________________________ >Boost-users mailing list >Boost-users(a)lists.boost.org >http://lists.boost.org/mailman/listinfo.cgi/boost-users Victor A. Wagner Jr. http://rudbek.com The five most dangerous words in the English language: "There oughta be a law"

1 0

Re: [Boost-users] Tokenizer Question
by Victor A. Wagner Jr. 12 Jun '05

12 Jun '05

At 10:52 2005-06-11, Tom Browder wrote: >I have used my own C++ tokenizer in the past, but I would like to use >Boost's instead. > >The predominant use of tokenizing for me is to split on white space, for tokenizing on whitespace, simple stream input (>>) to a std::string suffices. IMO, it's hardly worth troubling yourself with a tokenizer for whitespace. > but >Boost's default is to use white space AND punctuation. Is there any >possibility to have either the default changed, or another TokenizerFunction >added such as ws_separator, or something similar? > >I know I can use > > boost::char_separator<char> sep(" \n\t"); > >(but do I need to add "\v" to the char set?) > >but I would rather have something like > > boost::ws_separator sep; > >and, better, make the ws_separator be the default TokenizerFunction for >tokenizer. > >Thanks for listening. > >Tom Browder > > > > > > >_______________________________________________ >Boost-users mailing list >Boost-users(a)lists.boost.org >http://lists.boost.org/mailman/listinfo.cgi/boost-users Victor A. Wagner Jr. http://rudbek.com The five most dangerous words in the English language: "There oughta be a law"

2 1

RE: [Boost-users] Using minimal boost header files.
by Brian Braatz 11 Jun '05

11 Jun '05

I have never used this tool. But I believe this is what you are looking for: http://www.boost.org/tools/bcp/bcp.html Brian ________________________________ From: boost-users-bounces(a)lists.boost.org [mailto:boost-users-bounces@lists.boost.org] On Behalf Of Junkeh Sent: Friday, June 10, 2005 11:07 AM To: boost-users(a)lists.boost.org Subject: [Boost-users] Using minimal boost header files. Dear Sirs/Madams, I would like to be able to use some of the features of the boost library but only a very tiny portion of them! May I ask whether it is possible to do this and therefore distribute much less than 17Mbs worth of mostly unused code? Yours, Dan.

2 1

Building Boost on Mac OS X
by Brian Campbell 10 Jun '05

10 Jun '05

I'm trying to build boost (boost_1_31_0) as a dependency for monotone <http://venge.net/monotone/> on Mac OS X 10.4 (tiger). I'm doing this using gcc-3.3 because monotone doesn't work properly with gcc 4. Following the instructions in monotone for how to build boost, it tells me to do this: $BJAM "-sBUILD=release <threading>single <optimization>speed <runtime- link>static" This failed on a few of the targets, with an error that crt0.o was not found. This is probably because static linking of binaries is not supported on Mac OS X: http://developer.apple.com/qa/qa2001/qa1118.html So I tried removing the <runtime-link>static option, but then I started getting errors about not being able to find certain symbols. For some projects, it was complaining about not being able to find _main. It also complained about not being able to find __Z20init_unit_test_suiteiPPc in libboost_unit_test_framework, and there may have been other errors too. Here's the error output for some of these: gcc-Link-action bin/boost/libs/thread/build/libboost_thread.dylib/gcc/ release/shared-linkable-true/libboost_thread-gcc-1_31.dylib c++: unrecognized option `-shared' ld: Undefined symbols: _main gcc-Link-action bin/boost/libs/test/build/ libboost_unit_test_framework.dylib/gcc/release/shared-linkable-true/ libboost_unit_test_framework-gcc-1_31.dylib c++: unrecognized option `-shared' ld: Undefined symbols: __Z20init_unit_test_suiteiPPc So, I'm wondering what I need to do to get boost to build on darwin. Can anyone give me some help? Thanks, Brian

3 3

Boost.mpl metafunction with integral value problem
by Vyacheslav Kononenko 10 Jun '05

10 Jun '05

Hi, I've been trying boost::mpl for my current project and hit a problem. Sorry if it is an already known issue. So if we have our own metafucntion: template< typename T, typename Value > func : mpl::void_ {}; it works fine: typedef mpl::find_if< mpl::vector< int, char >, func< mpl::_1, mpl::int_< 10 > >::type iter; ... But if I try to use integral value instead: template< typename T, int Value > func : mpl::void_ {}; and typedef mpl::find_if< mpl::vector< int, char >, func< mpl::_1, 10 >::type iter; It fails on gcc 3.3.3 nor 3.4.3. So is it a limitation on language or compiler or library? I mean should second variant compile in generic? Regards, Vyacheslav

3 3

Re: [Boost-users] Re: Building boostbook from cvs fails lately (FIX).
by Peter Whaite 10 Jun '05

10 Jun '05

Peter Whaite <boost(a)whaite.ca> wrote: > Vladimir Prus <ghost(a)cs.msu.su> wrote: > > > > ../tools/build/v2/doc/src/install.xml:11: error: xmlParseEntityRef: > > > expecting ';' > > > > > url="http://sourceforge.net/project/showfiles.php?group_id=7586&package_i > > > > Well, that's unquoted '&' character in XML source. Fixed now, thanks > > for the report! > > Thanks -- that fixes the first error. Now I'm left with the second, > xsl:element : no namespace bound to prefix xi. I found some time to dig into this a little deeper. Turns out its due to these not being loaded > warning: failed to load external entity "src/hash.xml" > XInclude: could not load src/hash.xml, and no fallback was found > warning: failed to load external entity "src/static_assert.xml" > XInclude: could not load src/static_assert.xml, and no fallback was found > warning: failed to load external entity "src/type_traits.xml" > XInclude: could not load src/type_traits.xml, and no fallback was found and thats because <xi:include href="hash.xml"/> in src/boost.xml causes xsltproc to search for "./src/hash.xml", and not "./hash.xml", in the directories specified by --path on the command line. Including "../hash.xml" fixes it. The following patch to boost.xml works for me, but I dont really know enough about the boost docbook to know if its the right approach. % cvs diff boost.xml Index: boost.xml =================================================================== RCS file: /cvsroot/boost/boost/doc/src/boost.xml,v retrieving revision 1.15 diff -u -r1.15 boost.xml --- boost.xml 17 Apr 2005 10:43:53 -0000 1.15 +++ boost.xml 10 Jun 2005 18:09:45 -0000 @@ -302,7 +302,7 @@ </libraryinfo> </library> - <xi:include href="hash.xml"/> + <xi:include href="../hash.xml"/> <library name="Graph" dirname="graph" html-only="1"> <libraryinfo> @@ -650,7 +650,7 @@ </libraryinfo> </library> - <xi:include href="static_assert.xml"/> + <xi:include href="../static_assert.xml"/> <xi:include href="../../libs/algorithm/string/doc/string_algo.xml"/> @@ -705,7 +705,7 @@ </libraryinfo> </library> - <xi:include href="type_traits.xml"/> + <xi:include href="../type_traits.xml"/> <library name="uBLAS" dirname="numeric/ublas" html-only="1"> <libraryinfo>

1 0

Using minimal boost header files.
by Junkeh 10 Jun '05

10 Jun '05

Dear Sirs/Madams, I would like to be able to use some of the features of the boost library but only a very tiny portion of them! May I ask whether it is possible to do this and therefore distribute much less than 17Mbs worth of mostly unused code? Yours, Dan.

1 0

Serialization of heterogeneous sequence using shared_ptr<void>
by Sean Kelly 10 Jun '05

10 Jun '05

Dear All, I'm trying to figure out if something like the below is possible with the serialization library. The idea is to serialize a heterogeneous sequence using shared_pointer<void>. thanks for any help Sean void hetero_sequence() { // create and open a character archive for output std::ofstream ofs("dictionary.pickle"); boost::archive::text_oarchive oa(ofs); std::deque<boost::shared_ptr<void> > sequence; sequence.push_back(boost::shared_ptr<string>(new string("random string"))); sequence.push_back(boost::shared_ptr<double>(new double(3.1415627))); sequence.push_back(boost::shared_ptr<bool>(new bool(false))); // write class instance to archive oa << sequence; // close archive ofs.close(); }

3 3

boost/limits.hpp and GCC 3.2.3
by Dave Steffen 10 Jun '05

10 Jun '05

Folks, It looks to me like numeric_limits on GCC 3.2.3 is broken; specifically, numeric_limits<double>::has_infinity = false numeric_limits<double>::infinity() == 0 I suspect boost/limits.hpp could fix this if it knew to do so; but since <limits> exists, boost/limits.hpp doesn't 'fix' anything. Does anyone have thoughts about (reasonably portable) ways to "convince" boost/limits.hpp to provide numeric_limits even if the compiler already has one? Thanks! ---------------------------------------------------------------------- Dave Steffen, Ph.D. "There are two ways to write error-free Software Engineer IV programs; only the third one works." Numerica Corporation ph (970) 419-8343 x27 "Pie are not square. Pie are round. fax (970) 223-6797 Cornbread are square" dgsteffen(a)numerica.us ... anon (usenet)

1 0

Boost.Serialization and Pointers to Objects of Derived Classes
by Richard Jennings 10 Jun '05

10 Jun '05

I am using Boost 1.32.0 with Borland C++ Builder 6 and attempting to serialize an object via a pointer to a base class but I am getting a runtime exception of "unregistered void cast". Having read the page in the documentation on Runtime Casting, I thought I should be able to use boost::serialization::void_cast_register<Base, Derived>(0,0); to associate the base and derived classes, but I can't get it to work. If I use the alternate method (to serialize the base class and provide a do-nothing serialize method in the base class) then it works fine. Any ideas what's wrong? The program below demonstrates the problem. Class A uses the serialize base class method and that's ok while class B uses the void_cast_register method and an exception is thrown when it's serialized: ----- #include <iostream> #include <boost/filesystem/fstream.hpp> #include <boost/filesystem/path.hpp> #include <boost/archive/xml_oarchive.hpp> #include <boost/serialization/export.hpp> #include <boost/serialization/base_object.hpp> #if defined(_DEBUG) # pragma link "libboost_serialization-bcb-mt-d-1_32.lib" #else # pragma link "libboost_serialization-bcb-mt-1_32.lib" #endif class Base_c { public: virtual void SomeMethod() = 0; private: friend class boost::serialization::access; template<class Archive> void serialize(Archive & ar, const unsigned int version) {} }; BOOST_IS_ABSTRACT(Base_c) class A : public Base_c { public: A() : m_Id("A"){}; void SomeMethod(){}; private: friend class boost::serialization::access; template<class Archive> void serialize(Archive & ar, const unsigned int version) { /* boost::serialization::void_cast_register<Base_c, A>(0,0); */ ar & boost::serialization::make_nvp("Base", boost::serialization::base_object<Base_c>(*this)); ar & BOOST_SERIALIZATION_NVP(m_Id); } std::string const m_Id; }; BOOST_CLASS_EXPORT(A); class B : public Base_c { public: B() : m_Id("B"){}; void SomeMethod(){}; private: friend class boost::serialization::access; template<class Archive> void serialize(Archive & ar, const unsigned int version) { boost::serialization::void_cast_register<Base_c, B>(0,0); /* ar & boost::serialization::make_nvp("Base", boost::serialization::base_object<Base_c>(*this)); */ ar & BOOST_SERIALIZATION_NVP(m_Id); } std::string const m_Id; }; BOOST_CLASS_EXPORT(B) main(int, char**) { boost::filesystem::ofstream OfStream(boost::filesystem::path("D:\\Temp\\SerialisationText.xml", boost::filesystem::native)); boost::archive::xml_oarchive Archive(OfStream); try { Base_c *pBase, *pBase2; pBase = new A(); pBase2 = new B(); Archive & boost::serialization::make_nvp("AByBasePointer", pBase); std::cout << "Serialised A ok\n"; Archive & boost::serialization::make_nvp("BByBasePointer", pBase2); std::cout << "Serialised B ok\n"; } catch (std::exception& E) { std::cout << "Exception: " << E.what(); } } ----- The output is: Serialised A ok Exception: unregistered void cast

3 2