August 2011 - Boost-users - lists.preview.boost.org

[Regex | Xpressive] Efficiently "grepping" large files
by Thomas Luzat 17 Aug '11

17 Aug '11

Hello, I am looking into Boost.Regex and Boost.Xpressive for implementing grep-like functionality on large files. More specifically I want to check whether a file contains a match for an expression which does not need that much context (a few KB). regex_search and BidirectionalIterators for the whole file work, but there is a complication: I want to do other operations on the file in other threads in parallel (such as calculating hashes and extracting other data). Given that this is mostly I/O bound I would like to avoid rereading portions of the file as much as possible. My idea was to implement a circular buffer which may block the workers (and reading, if data is ever read faster than analyzed). The circular buffer needs to know when old data is no longer required, though, that is regex_search no longer works on it or backtracks into it. The best idea so far is to read blocks of data and do slightly overlapped calls of regex_search on those blocks. This approach has got at least two problems: 1) Matches and context are potentially limited to the size of the overlap. 2) Semantics of the match might be changed if I choose an arbitrary cut for the overlap (e.g. ^ and $ might change because I cut the beginning or end of a line) I would not mind to specify a maximum context of a few KB to maybe 1 MB, but I do not want to change semantics. Is there a better approach? I was thinking about supplying some intelligent iterators, but Regex/Xpressive might leave copies around longer than I anticipate, such that I still cannot decide where they are and how far they may still backtrack. If that's not really possible with the boost libraries I would be interested in alternatives, too. I am already considering looking into some grep implementations. Thanks in advance! Thomas Luzat

3 5

cross-compiling didn't work with boost:filesystem
by rudy b 17 Aug '11

17 Aug '11

Hi, I developed a code on Linux, and the only library that I used was boost:filesystem. I was very happy with the code on Linux, and it never gave me a problem and never broke on me, I tried it on different scenarios and different file types, and I had never issue with boost::filesystem::remove(&path) function. But, now, I needed my code on a Mac machine (but I don't have a Mac) so I borrowed my friend's Mac, and installed xcode, and installed boost_1.47, and I didn't get a single error or warning when installing these packages. As a test, I even ran some simple filesystem examples that were provided. and they all passed. I even used the same "make" file that I had on linux, and it worked great on Mac. (no warnings when compiled) But, when I tried to run my code, I was getting errors with ::remove functions. I am not sure, but I think part of it was because I may have read-only permission on some of the files, and maybe that is why it complained, although I never had this problem in Linux. Also, another part of it could be because of the file naming with white space between its characters: (e.g. "my document.txt" because of this space). I don't know, but I thought that "boost::filesystem" should be independent of the developing platform, and that I could be fairly confident that if it works on one platform (Linux), it should be working on another one (Mac). Any idea, what I am doing wrong, and how this can be fixed? thanks, RB

2 1

[Containers Library Review] The review of the Containers library by Ion Gaztanaga starts today
by John Maddock 17 Aug '11

17 Aug '11

The review of the containers library by Ion Gaztanaga starts today, to summarize the library: "Boost.Container library implements several well-known containers, including STL containers. The aim of the library is to offers advanced features not present in standard containers or to offer the latest standard draft features for compilers that comply with C++03. In short, what does Boost.Container offer? * Move semantics are implemented, including move emulation for pre-C++0x compilers. * New advanced features (e.g. placement insertion, recursive containers) are present. * Containers support stateful allocators and are compatible with Boost.Interprocess (they can be safely placed in shared memory). * The library offers new useful containers: o flat_map, flat_set, flat_multiset and flat_multiset: drop-in replacements for standard associative containers but more memory friendly and with faster searches. o stable_vector: a std::list and std::vector hybrid with random-access iterators that offers iterator stability in insertions and erasures. o slist: the classic pre-standard singly linked list container." Documentation from the library may be viewed online here: file:///M:/data/boost/sandbox/move/libs/container/doc/html/index.html The source may be accessed from the "move" directory of the sandbox SVN or downloaded from http://www.drivehq.com/web/igaztanaga/boost_container.zip. Note that this download contains a copy of the accepted, but not yet release, Boost.Move library - extract the zip over a copy of Boost-1.47 to get a full working copy. Review comments might like to answer the following questions: * What is your evaluation of the design? * What is your evaluation of the implementation? * What is your evaluation of the documentation? * What is your evaluation of the potential usefulness of the library? * Did you try to use the library? With what compiler? Did you have any problems? * How much effort did you put into your evaluation? A glance? A quick reading? In-depth study? * Are you knowledgeable about the problem domain? And finally, every review should answer this question: * Do you think the library should be accepted as a Boost library? Be sure to say this explicitly so that your other comments don't obscure your overall opinion. Regards, John Maddock.

5 5

ublas bug?
by Peter Man 17 Aug '11

17 Aug '11

Hi, I was just wondering if there is a bug in ublas (boost version 1.47): when I use the following code (in debug mode): ublas::matrix<double> m; ublas::matrix<double> m2; m2 = m; this compiles in MSVC 10, but produces a runtime error "Expression: invalid null pointer". I had a look at some of the underlying ublas code, and found that default initialisation of matrices makes the underlying data pointer "data_" be set to NULL, and that when one matrix is copied to another, std::copy is invoked to copy one data_ to the other. However, I looked at the MSVC std::copy definition, and it seems to check for NULLness of the pointers. The following link explains this in more detail: https://connect.microsoft.com/VisualStudio/feedback/details/558044/std-copy… On the plus side, I have no such problem when running the above code in linux. Thanks, Peter

1 0

Simple inclusion of (1.47.0) filesystem.hpp generates compiler errors.
by Bill Moo 17 Aug '11

17 Aug '11

Hello Users, Qt v4.7.2 Boost v1.47.0. Intel Compiler v 12.0.1029.2010 Windows 7 Ultimate N x64 Building an x64 Release binary I have just included the boost/filesystem.hpp file in to my application and re-compiled only for the compiler to generate these errors. As yet I am not using any of the functionality of this I just included the file. Is this another Intel / MSVC STL issue? Sorry for the dump but without it I assume you won't be able to help. ..\Includes\boost/filesystem/v3/operations.hpp(126): error: expected either a definition or a tag name BOOST_SCOPED_ENUM_START(copy_option) ^ ..\Includes\boost/filesystem/v3/operations.hpp(127): error: expected an identifier {none, fail_if_exists = none, overwrite_if_exists}; ^ ..\Includes\boost/filesystem/v3/operations.hpp(127): error: expected a ";" {none, fail_if_exists = none, overwrite_if_exists}; ^ ..\Includes\boost/filesystem/v3/operations.hpp(150): error: identifier "copy_option" is undefined BOOST_SCOPED_ENUM(copy_option) option, // See ticket #2925 ^ ..\Includes\boost/filesystem/v3/operations.hpp(299): error: identifier "copy_option" is undefined BOOST_SCOPED_ENUM(copy_option) option) ^ ..\Includes\boost/filesystem/v3/operations.hpp(303): error: name followed by "::" must be a class or namespace name {detail::copy_file(from, to, copy_option::fail_if_exists);} ^ ..\Includes\boost/filesystem/v3/operations.hpp(306): error: identifier "copy_option" is undefined BOOST_SCOPED_ENUM(copy_option) option, system::error_code& ec) ^ ..\Includes\boost/filesystem/v3/operations.hpp(310): error: name followed by "::" must be a class or namespace name {detail::copy_file(from, to, copy_option::fail_if_exists, &ec);} ^ ..\Includes\boost/filesystem/v3/operations.hpp(642): error: expected either a definition or a tag name BOOST_SCOPED_ENUM_START(symlink_option) ^ ..\Includes\boost/filesystem/v3/operations.hpp(643): error: expected an identifier { ^ ..\Includes\boost/filesystem/v3/operations.hpp(643): error: expected a ";" { ^ ..\Includes\boost/filesystem/v3/operations.hpp(648): warning #12: parsing restarts here after previous syntax error }; ^ ..\Includes\boost/filesystem/v3/operations.hpp(651): error: identifier "symlink_option" is undefined BOOST_BITMASK(BOOST_SCOPED_ENUM(symlink_option)) ^ ..\Includes\boost/filesystem/v3/operations.hpp(651): error: identifier "symlink_option" is undefined BOOST_BITMASK(BOOST_SCOPED_ENUM(symlink_option)) ^ ..\Includes\boost/filesystem/v3/operations.hpp(651): error: identifier "symlink_option" is undefined BOOST_BITMASK(BOOST_SCOPED_ENUM(symlink_option)) ^ ..\Includes\boost/filesystem/v3/operations.hpp(651): error: nonmember operator requires a parameter with class or enum type BOOST_BITMASK(BOOST_SCOPED_ENUM(symlink_option)) ^ ..\Includes\boost/filesystem/v3/operations.hpp(651): error: identifier "symlink_option" is undefined BOOST_BITMASK(BOOST_SCOPED_ENUM(symlink_option)) ^ The two errors above are repeated and repeated many times so I removed then to save bandwidth, space and time. ..\Includes\boost/filesystem/v3/operations.hpp(662): error: name followed by "::" must be a class or namespace name recur_dir_itr_imp() : m_level(0), m_options(symlink_option::none) {} ^ ..\Includes\boost/filesystem/v3/operations.hpp(678): error: name followed by "::" must be a class or namespace name if ((m_options & symlink_option::_detail_no_push) == symlink_option::_detail_no_push) ^ ..\Includes\boost/filesystem/v3/operations.hpp(678): error: name followed by "::" must be a class or namespace name if ((m_options & symlink_option::_detail_no_push) == symlink_option::_detail_no_push) ^ ..\Includes\boost/filesystem/v3/operations.hpp(679): error: name followed by "::" must be a class or namespace name m_options &= ~symlink_option::_detail_no_push; ^ ..\Includes\boost/filesystem/v3/operations.hpp(682): error: name followed by "::" must be a class or namespace name || (m_options & symlink_option::recurse) == symlink_option::recurse)) ^ ..\Includes\boost/filesystem/v3/operations.hpp(682): error: name followed by "::" must be a class or namespace name || (m_options & symlink_option::recurse) == symlink_option::recurse)) ^ ..\Includes\boost/filesystem/v3/operations.hpp(737): error: identifier "symlink_option" is undefined BOOST_SCOPED_ENUM(symlink_option) opt = symlink_option::none) ^ ..\Includes\boost/filesystem/v3/operations.hpp(737): error: name followed by "::" must be a class or namespace name BOOST_SCOPED_ENUM(symlink_option) opt = symlink_option::none) ^ ..\Includes\boost/filesystem/v3/operations.hpp(747): error: identifier "symlink_option" is undefined BOOST_SCOPED_ENUM(symlink_option) opt, ^ ..\Includes\boost/filesystem/v3/operations.hpp(737): error: name followed by "::" must be a class or namespace name BOOST_SCOPED_ENUM(symlink_option) opt = symlink_option::none) ^ ..\Includes\boost/filesystem/v3/operations.hpp(761): error: name followed by "::" must be a class or namespace name m_imp->m_options = symlink_option::none; ^ ..\Includes\boost/filesystem/v3/operations.hpp(783): error: name followed by "::" must be a class or namespace name return (m_imp->m_options & symlink_option::_detail_no_push) ^ ..\Includes\boost/filesystem/v3/operations.hpp(784): error: name followed by "::" must be a class or namespace name == symlink_option::_detail_no_push; ^ ..\Includes\boost/filesystem/v3/operations.hpp(802): error: name followed by "::" must be a class or namespace name m_imp->m_options |= symlink_option::_detail_no_push; ^ ..\Includes\boost/filesystem/v3/operations.hpp(804): error: name followed by "::" must be a class or namespace name m_imp->m_options &= ~symlink_option::_detail_no_push; ^ ..\Includes\boost/filesystem/v3/operations.hpp(988): error: namespace "boost::filesystem3" has no member "copy_option" using filesystem3::copy_option; ^ ..\Includes\boost/filesystem/v3/operations.hpp(1031): error: namespace "boost::filesystem3" has no member "symlink_option" using filesystem3::symlink_option; ^ -- Bill

1 0

[Container] Forward declaration of movable types
by Szymon Gatner 17 Aug '11

17 Aug '11

Hi, I am evaluating container library by Ion Gaztanaga. I know it is not officially a part of boost yet but I didn't know where else to ask this question. My problem: If there is a movable class 'MyMovable' (I use boost.move) it can be stored without unnecessary copies in move-aware vector that is declared like so: vector<MyMovable> movables; which is very cool until this vector is suppose be a member of other class like: //widget.hpp class Widget { vector<MyMovable> movables_; }; which is not cool at all because now widget.hpp must include my_movable.hpp as opposed to just forward declaring it if it stored say vector<shared_ptr<MyMovable>> which in turn makes movability of MyMovable rather useless. I guess my question is: how can one take advantage of move semantics in context of member containers without adding unnecessary class dependencies? Regards, Simon

2 3

About Bug #4978 (deadlock on interrupt)
by Fabien Chêne 17 Aug '11

17 Aug '11

Hello, The bug #4978 seems to be fixed, according to its status. However, is it really fixed in the release 1.47, as indicated by a comment at the end ? I cannot see anything about that in the release note for 1.47. I think that it would be worth mentioning it. Regards, -- Fabien

1 0

Re: [Boost-users] [containers review] A reminder that the containers review ends tomorrow
by John Maddock 17 Aug '11

17 Aug '11

The containers library review is now officially over. However, there are still a few drips a drabs of reviews coming in.... are there any more still pending? Thanks, John.

1 0

[MPI][Boost] Some Questions About Delta Stepping Dijkstra in Boost
by Chen Kai 17 Aug '11

17 Aug '11

Environment: Boost1.47 + Microsoft HPC 2008 SDK I find Delta Stepping Dijkstra and Crauser Dijkstra in boost, and both of them are slower than sequential dijkstra in boost. (About 10 times slower. I used 10 graphs, the biggest graph contains 10 million nodes and 10 million edges) Q1: I wonder why the parallel dijkstra is slower than sequential version? Q2: Can I adjust the number of threads that Delta Stepping used? (For exemple, I want it use 1, 2, 3, 4 threads respectively. And Can I make it sequential version?) Thx --- Chen Kai School of Computer Science, Fudan University blog: http://www.cnblogs.com/remlostime/

1 0

problems with installing boost python
by Wilson Tan 17 Aug '11

17 Aug '11

Hello everyone, I had been trying to build BOOST for SST, but so far I had been unsuccessful with the Python(and IOSTREAM) components. I followed Section 3.1( http://www.boost.org/doc/libs/1_47_0/libs/python/doc/building.html) but even that is not fully successful: wilson@vorpalsword:~/SST/boost_1_47_0$ ./bjam --build-dir=~/SST/boost_1_47_0/libs/python/example/quickstart toolset=gcc --verbose-test test notice: could not find main target test notice: assuming it is a name of file to create. Building the Boost C++ Libraries. Performing configuration checks - has_icu builds : no - ../config//has_gcc_visibility builds : yes - ../config//has_long_double_support builds : yes don't know how to make <e>test ...found 1 target... ...can't find 1 target... wilson@vorpalsword:~/SST/boost_1_47_0$ The first lines of my Python related errors while installing are the following ...skipped <p/usr/local/lib>libboost_iostreams.so for lack of <p/usr/local/lib>libboost_iostreams.so.1.47.0... gcc.compile.c++ bin.v2/libs/python/build/gcc-4.4.3/release/threading-multi/numeric.o In file included from ./boost/python/detail/prefix.hpp:13, from ./boost/python/numeric.hpp:8, from libs/python/src/numeric.cpp:6: ./boost/python/detail/wrap_python.hpp:75:24: error: patchlevel.h: No such file or directory ./boost/python/detail/wrap_python.hpp:78:2: error: #error Python 2.2 or higher is required for this version of Boost.Python. ./boost/python/detail/wrap_python.hpp:142:21: error: Python.h: No such file or directory ./boost/python/detail/wrap_python.hpp:203:4: error: #error "HAVE_LONG_LONG defined but not PY_LONG_LONG or LONG_LONG" In file included from ./boost/python/object/pointer_holder.hpp:14, from ./boost/python/to_python_indirect.hpp:10, from ./boost/python/converter/arg_to_python.hpp:10, Result of whereis: wilson@vorpalsword:~/SST/boost_1_47_0$ whereis python python: /usr/bin/python2.6 /usr/bin/python /etc/python2.6 /etc/python /usr/lib/python3.1 /usr/lib/python2.6 /usr/local/lib/python2.6 /usr/include/python2.6 /usr/share/python /usr/share/man/man1/python.1.gz For good measure, I also specified the interpreter location in project-config.jam. # Python configuration using python : 2.6 : /usr/bin/python ; using mpi ; Could somebody give me pointers on how to fix this and get Python running on Boost? I'd really appreciate it.

2 2