- Boost - lists.preview.boost.org

[mpl] #861 Detect nested template members with has_template_xxx
by Steven Watanabe 09 Jun '10

09 Jun '10

AMDG The patch attached to https://svn.boost.org/trac/boost/ticket/861 works with all the compilers I have to test with. It's fairly long, so I'm not including it here. Okay to commit? In Christ, Steven Watanabe

3 4

Bug Sprint daily update for Monday, June 7th
by Marshall Clow 08 Jun '10

08 Jun '10

As of 0800 Pacific time today, we had 958 open trac tickets. That is down by 30 from yesterday, and 31 since the start of the bug sprint. Great work, everyone! Thanks especially to Steven W, Andy W, and Viboes for all their work. Activity summary for yesterday: New tickets: 3 Modified: 38 Closed: 34 Reopened: 1 Today, rather than the summary totals for yesterday, I have deltas; changes in the totals. If you'ld like to see the totals, they are available online at: Open ticket count by owner: https://svn.boost.org/trac/boost/report/18 Open ticket count by component: https://svn.boost.org/trac/boost/report/19 Open ticket count by type https://svn.boost.org/trac/boost/report/20 (Overwhelmingly, these are 'bugs') Open ticket count by milestone: https://svn.boost.org/trac/boost/report/21 (All changes are since the start of the bug sprint; 0800 PDT Saturday June 5th) Changed totals by library: None -16 thread -12 Building Boost -2 date_time -2 GIL -1 asio -1 filesystem -1 function -1 fusion -1 iterator -1 quickbook -1 Documentation 1 circular_buffer 1 foreach 1 interprocess 1 intrusive 1 mpi 1 mpl 1 pool 1 program_options 1 serialization 1 Python 2 utility 2 build 5 Changed totals by author: <none> -25 anthonyw -11 djowel -6 az_sw_dude -2 bemandawes -1 bschaeling -1 chris_kohlhoff -1 dlwalker -1 hljin -1 speedsnail -1 agurtovoy 1 cnewbold 1 danielw 1 eric_niebler 1 jano_gaspar 1 matias 1 niels_dekker 1 no-maintainer 1 pdimov 1 ramey 1 witt 1 igaztanaga 2 danieljames 4 vladimir_prus 9 Changed totals by category: Bugs -17 Feature Requests -5 Patches 1 Support Requests 1 -- Marshall

3 4

[regex, xpressive] interesting(?) perf benchmark
by Eric Niebler 08 Jun '10

08 Jun '10

I'm always suspect of benchmarks, but I figured I'd post this anyway. http://astl.sourceforge.net/bench.7.html. It seems someone wrote a generic automata library and a regex engine on top of it, and compared its performance to boost.regex, xpressive, pcre, and greta. Of course, his library wins handily. (The purpose of benchmarking is to make yourself look good, right? ;-) He doesn't say that (a) his engine is a DFA, and he's comparing to NFAs; and (b) he's only implemented a tiny subset of regex features people would expect; e.g., no captures even, let alone backreferences: things that can be tricky or impossible to implement with DFAs. He also doesn't say what version of Boost he's using (tests run Nov. 2009) or whether he's using static or dynamic xpressive (probably dynamic; static can be ~15% faster). Anyway, it's not my intention to kick off yet another costly escalation in the regex war. God help me, I don't have the time. I post it here because I haven't come across a comparison of these four libraries by a third party before. Some quick observations: - I'm pleased that on the balance, xpressive fares well here relative to the other NFAs. In the end, though, not much separates best from worst. - All the NFA libraries seem to have large overhead that becomes very noticeable when matching short strings. I don't know why. - I'm gobsmacked that my VERY OLD library greta handily beats pcre, boost.regex /and/ xpressive on short matches. Failing to beat yourself in any perf benchmark is kind of embarrassing. - I'm tickled that for long matches, both xpressive and boost.regex (NFAs) beat his library (DFA). How'd that happen? - I find it interesting that both xpressive and boost.regex have large discontinuities in the chart at the bottom; the others don't. Huh. When more std library vendors start shipping regex libraries, I'll be interested in seeing this perf matrix expanded. -- Eric Niebler BoostPro Computing http://www.boostpro.com

10 22

Spam in ticket 865
by Jeremiah Willcock 08 Jun '10

08 Jun '10

There is a spam comment in <URL:https://svn.boost.org/trac/boost/ticket/865> that should be removed. -- Jeremiah Willcock

1 0

[asio] Race conditions detected with valgrind drd in example/http/server2
by Jorge Moraleda 08 Jun '10

08 Jun '10

When testing boost.asio (example/http/server2) for race conditions using valgrind drd, some errors are reported right after the first connection to the server is made. I am using Valgrind-3.6.0.SVN and gcc version 4.4.4 (Debian 4.4.4-4). This is an up to date Debian sid 64 bit system. To reproduce: g++ -l boost_thread -l boost_system /opt/boost/boost/doc/html/boost_asio/example/http/server2/*cpp valgrind --tool=drd ./a.out 127.0.0.1 31174 8 /tmp While it is running type "http://localhost:31174/main.cpp" in your browser (where main.cpp is a file in /tmp). Then several race conditions occur. These are the valgrind error messages: ==7737== drd, a thread error detector ==7737== Copyright (C) 2006-2010, and GNU GPL'd, by Bart Van Assche. ==7737== Using Valgrind-3.6.0.SVN and LibVEX; rerun with -h for copyright info ==7737== Command: ./a.out 127.0.0.1 31174 8 /tmp ==7737== ==7737== Thread 4: ==7737== Conflicting load by thread 4 at 0x061875f0 size 8 ==7737== at 0x40DB72: boost::asio::detail::op_queue<boost::asio::detail::reactor_op>::front() (in /tmp/a.out) ==7737== by 0x417974: boost::asio::detail::epoll_reactor::run(bool, boost::asio::detail::op_queue<boost::asio::detail::task_io_service_operation<boost::asio::detail::epoll_reactor> >&) (in /tmp/a.out) ==7737== by 0x41C809: boost::asio::detail::task_io_service<boost::asio::detail::epoll_reactor>::do_one(boost::asio::detail::scoped_lock<boost::asio::detail::posix_mutex>&, boost::asio::detail::task_io_service<boost::asio::detail::epoll_reactor>::idle_thread_info*) (in /tmp/a.out) ==7737== by 0x41A026: boost::asio::detail::task_io_service<boost::asio::detail::epoll_reactor>::run(boost::system::error_code&) (in /tmp/a.out) ==7737== by 0x417D26: boost::asio::io_service::run() (in /tmp/a.out) ==7737== by 0x430358: unsigned long boost::_mfi::mf0<unsigned long, boost::asio::io_service>::call<boost::shared_ptr<boost::asio::io_service> >(boost::shared_ptr<boost::asio::io_service>&, void const*) const (in /tmp/a.out) ==7737== by 0x42FF74: unsigned long boost::_mfi::mf0<unsigned long, boost::asio::io_service>::operator()<boost::shared_ptr<boost::asio::io_service> >(boost::shared_ptr<boost::asio::io_service>&) const (in /tmp/a.out) ==7737== by 0x42FE1E: unsigned long boost::_bi::list1<boost::_bi::value<boost::shared_ptr<boost::asio::io_service> > >::operator()<unsigned long, boost::_mfi::mf0<unsigned long, boost::asio::io_service>, boost::_bi::list0>(boost::_bi::type<unsigned long>, boost::_mfi::mf0<unsigned long, boost::asio::io_service>&, boost::_bi::list0&, long) (in /tmp/a.out) ==7737== by 0x42FD66: boost::_bi::bind_t<unsigned long, boost::_mfi::mf0<unsigned long, boost::asio::io_service>, boost::_bi::list1<boost::_bi::value<boost::shared_ptr<boost::asio::io_service> > > >::operator()() (in /tmp/a.out) ==7737== by 0x42EE79: boost::detail::thread_data<boost::_bi::bind_t<unsigned long, boost::_mfi::mf0<unsigned long, boost::asio::io_service>, boost::_bi::list1<boost::_bi::value<boost::shared_ptr<boost::asio::io_service> > > > >::run() (in /tmp/a.out) ==7737== by 0x4E41F5F: thread_proxy (in /usr/local/lib/libboost_thread.so.1.43.0) ==7737== by 0x4C2EA60: vgDrd_thread_wrapper (drd_pthread_intercepts.c:272) ==7737== Address 0x61875f0 is at offset 80 from 0x61875a0. Allocation context: ==7737== at 0x4C26FD1: operator new(unsigned long) (vg_replace_malloc.c:261) ==7737== by 0x43BDC8: __gnu_cxx::new_allocator<std::_List_node<std::pair<int, boost::asio::detail::epoll_reactor::descriptor_state> > >::allocate(unsigned long, void const*) (in /tmp/a.out) ==7737== by 0x43B72D: std::_List_base<std::pair<int, boost::asio::detail::epoll_reactor::descriptor_state>, std::allocator<std::pair<int, boost::asio::detail::epoll_reactor::descriptor_state> > >::_M_get_node() (in /tmp/a.out) ==7737== by 0x43AD64: std::list<std::pair<int, boost::asio::detail::epoll_reactor::descriptor_state>, std::allocator<std::pair<int, boost::asio::detail::epoll_reactor::descriptor_state> > >::_M_create_node(std::pair<int, boost::asio::detail::epoll_reactor::descriptor_state> const&) (in /tmp/a.out) ==7737== by 0x43A0FC: std::list<std::pair<int, boost::asio::detail::epoll_reactor::descriptor_state>, std::allocator<std::pair<int, boost::asio::detail::epoll_reactor::descriptor_state> > >::insert(std::_List_iterator<std::pair<int, boost::asio::detail::epoll_reactor::descriptor_state> >, std::pair<int, boost::asio::detail::epoll_reactor::descriptor_state> const&) (in /tmp/a.out) ==7737== by 0x439A77: boost::asio::detail::hash_map<int, boost::asio::detail::epoll_reactor::descriptor_state>::values_insert(std::_List_iterator<std::pair<int, boost::asio::detail::epoll_reactor::descriptor_state> >, std::pair<int, boost::asio::detail::epoll_reactor::descriptor_state> const&) (in /tmp/a.out) ==7737== by 0x438B60: boost::asio::detail::hash_map<int, boost::asio::detail::epoll_reactor::descriptor_state>::insert(std::pair<int, boost::asio::detail::epoll_reactor::descriptor_state> const&) (in /tmp/a.out) ==7737== by 0x43811B: boost::asio::detail::epoll_reactor::register_descriptor(int, boost::asio::detail::epoll_reactor::descriptor_state*&) (in /tmp/a.out) ==7737== by 0x43DE25: boost::asio::detail::reactive_socket_service<boost::asio::ip::tcp>::assign(boost::asio::detail::reactive_socket_service<boost::asio::ip::tcp>::implementation_type&, boost::asio::ip::tcp const&, int const&, boost::system::error_code&) (in /tmp/a.out) ==7737== by 0x43D954: boost::asio::stream_socket_service<boost::asio::ip::tcp>::assign(boost::asio::detail::reactive_socket_service<boost::asio::ip::tcp>::implementation_type&, boost::asio::ip::tcp const&, int const&, boost::system::error_code&) (in /tmp/a.out) ==7737== by 0x43D689: boost::asio::basic_socket<boost::asio::ip::tcp, boost::asio::stream_socket_service<boost::asio::ip::tcp> >::assign(boost::asio::ip::tcp const&, int const&, boost::system::error_code&) (in /tmp/a.out) ==7737== by 0x43D274: boost::asio::detail::reactive_socket_service<boost::asio::ip::tcp>::accept_op_base<boost::asio::basic_socket<boost::asio::ip::tcp, boost::asio::stream_socket_service<boost::asio::ip::tcp> > >::do_perform(boost::asio::detail::reactor_op*) (in /tmp/a.out) ==7737== Other segment start (thread 3) ==7737== at 0x4C2F283: pthread_mutex_lock (drd_pthread_intercepts.c:578) ==7737== by 0x40AC2B: boost::asio::detail::posix_mutex::lock() (in /tmp/a.out) ==7737== by 0x40D4CF: boost::asio::detail::scoped_lock<boost::asio::detail::posix_mutex>::scoped_lock(boost::asio::detail::posix_mutex&) (in /tmp/a.out) ==7737== by 0x4380D4: boost::asio::detail::epoll_reactor::register_descriptor(int, boost::asio::detail::epoll_reactor::descriptor_state*&) (in /tmp/a.out) ==7737== by 0x43DE25: boost::asio::detail::reactive_socket_service<boost::asio::ip::tcp>::assign(boost::asio::detail::reactive_socket_service<boost::asio::ip::tcp>::implementation_type&, boost::asio::ip::tcp const&, int const&, boost::system::error_code&) (in /tmp/a.out) ==7737== by 0x43D954: boost::asio::stream_socket_service<boost::asio::ip::tcp>::assign(boost::asio::detail::reactive_socket_service<boost::asio::ip::tcp>::implementation_type&, boost::asio::ip::tcp const&, int const&, boost::system::error_code&) (in /tmp/a.out) ==7737== by 0x43D689: boost::asio::basic_socket<boost::asio::ip::tcp, boost::asio::stream_socket_service<boost::asio::ip::tcp> >::assign(boost::asio::ip::tcp const&, int const&, boost::system::error_code&) (in /tmp/a.out) ==7737== by 0x43D274: boost::asio::detail::reactive_socket_service<boost::asio::ip::tcp>::accept_op_base<boost::asio::basic_socket<boost::asio::ip::tcp, boost::asio::stream_socket_service<boost::asio::ip::tcp> > >::do_perform(boost::asio::detail::reactor_op*) (in /tmp/a.out) ==7737== by 0x40AF7A: boost::asio::detail::reactor_op::perform() (in /tmp/a.out) ==7737== by 0x41798B: boost::asio::detail::epoll_reactor::run(bool, boost::asio::detail::op_queue<boost::asio::detail::task_io_service_operation<boost::asio::detail::epoll_reactor> >&) (in /tmp/a.out) ==7737== by 0x41C809: boost::asio::detail::task_io_service<boost::asio::detail::epoll_reactor>::do_one(boost::asio::detail::scoped_lock<boost::asio::detail::posix_mutex>&, boost::asio::detail::task_io_service<boost::asio::detail::epoll_reactor>::idle_thread_info*) (in /tmp/a.out) ==7737== by 0x41A026: boost::asio::detail::task_io_service<boost::asio::detail::epoll_reactor>::run(boost::system::error_code&) (in /tmp/a.out) ==7737== Other segment end (thread 3) ==7737== at 0x4C2FE7F: pthread_mutex_unlock (drd_pthread_intercepts.c:631) ==7737== by 0x40AC45: boost::asio::detail::posix_mutex::unlock() (in /tmp/a.out) ==7737== by 0x40D500: boost::asio::detail::scoped_lock<boost::asio::detail::posix_mutex>::~scoped_lock() (in /tmp/a.out) ==7737== by 0x438273: boost::asio::detail::epoll_reactor::register_descriptor(int, boost::asio::detail::epoll_reactor::descriptor_state*&) (in /tmp/a.out) ==7737== by 0x43DE25: boost::asio::detail::reactive_socket_service<boost::asio::ip::tcp>::assign(boost::asio::detail::reactive_socket_service<boost::asio::ip::tcp>::implementation_type&, boost::asio::ip::tcp const&, int const&, boost::system::error_code&) (in /tmp/a.out) ==7737== by 0x43D954: boost::asio::stream_socket_service<boost::asio::ip::tcp>::assign(boost::asio::detail::reactive_socket_service<boost::asio::ip::tcp>::implementation_type&, boost::asio::ip::tcp const&, int const&, boost::system::error_code&) (in /tmp/a.out) ==7737== by 0x43D689: boost::asio::basic_socket<boost::asio::ip::tcp, boost::asio::stream_socket_service<boost::asio::ip::tcp> >::assign(boost::asio::ip::tcp const&, int const&, boost::system::error_code&) (in /tmp/a.out) ==7737== by 0x43D274: boost::asio::detail::reactive_socket_service<boost::asio::ip::tcp>::accept_op_base<boost::asio::basic_socket<boost::asio::ip::tcp, boost::asio::stream_socket_service<boost::asio::ip::tcp> > >::do_perform(boost::asio::detail::reactor_op*) (in /tmp/a.out) ==7737== by 0x40AF7A: boost::asio::detail::reactor_op::perform() (in /tmp/a.out) ==7737== by 0x41798B: boost::asio::detail::epoll_reactor::run(bool, boost::asio::detail::op_queue<boost::asio::detail::task_io_service_operation<boost::asio::detail::epoll_reactor> >&) (in /tmp/a.out) ==7737== by 0x41C809: boost::asio::detail::task_io_service<boost::asio::detail::epoll_reactor>::do_one(boost::asio::detail::scoped_lock<boost::asio::detail::posix_mutex>&, boost::asio::detail::task_io_service<boost::asio::detail::epoll_reactor>::idle_thread_info*) (in /tmp/a.out) ==7737== by 0x41A026: boost::asio::detail::task_io_service<boost::asio::detail::epoll_reactor>::run(boost::system::error_code&) (in /tmp/a.out) ==7737== ==7737== Thread 3: ==7737== Probably a race condition: condition variable 0x8583c40 has been signaled but the associated mutex 0x61820f8 is not locked by the signalling thread. ==7737== at 0x4C283F9: pthread_cond_signal@* (drd_pthread_intercepts.c:736) ==7737== by 0x4119FE: void boost::asio::detail::posix_event::signal_and_unlock<boost::asio::detail::scoped_lock<boost::asio::detail::posix_mutex> >(boost::asio::detail::scoped_lock<boost::asio::detail::posix_mutex>&) (in /tmp/a.out) ==7737== by 0x4108F2: boost::asio::detail::task_io_service<boost::asio::detail::epoll_reactor>::wake_one_idle_thread_and_unlock(boost::asio::detail::scoped_lock<boost::asio::detail::posix_mutex>&) (in /tmp/a.out) ==7737== by 0x40F1C2: boost::asio::detail::task_io_service<boost::asio::detail::epoll_reactor>::wake_one_thread_and_unlock(boost::asio::detail::scoped_lock<boost::asio::detail::posix_mutex>&) (in /tmp/a.out) ==7737== by 0x40DA56: boost::asio::detail::task_io_service<boost::asio::detail::epoll_reactor>::init_task() (in /tmp/a.out) ==7737== by 0x40B941: boost::asio::detail::epoll_reactor::init_task() (in /tmp/a.out) ==7737== by 0x4148D8: boost::asio::detail::reactive_socket_service<boost::asio::ip::tcp>::reactive_socket_service(boost::asio::io_service&) (in /tmp/a.out) ==7737== by 0x414035: boost::asio::stream_socket_service<boost::asio::ip::tcp>::stream_socket_service(boost::asio::io_service&) (in /tmp/a.out) ==7737== by 0x413893: boost::asio::io_service::service* boost::asio::detail::service_registry::create<boost::asio::stream_socket_service<boost::asio::ip::tcp> >(boost::asio::io_service&) (in /tmp/a.out) ==7737== by 0x40ADB8: boost::asio::detail::service_registry::do_use_service(boost::asio::io_service::service::key const&, boost::asio::io_service::service* (*)(boost::asio::io_service&)) (in /tmp/a.out) ==7737== by 0x412B7E: boost::asio::stream_socket_service<boost::asio::ip::tcp>& boost::asio::detail::service_registry::use_service<boost::asio::stream_socket_service<boost::asio::ip::tcp> >() (in /tmp/a.out) ==7737== by 0x411B3A: boost::asio::stream_socket_service<boost::asio::ip::tcp>& boost::asio::use_service<boost::asio::stream_socket_service<boost::asio::ip::tcp> >(boost::asio::io_service&) (in /tmp/a.out) ==7737== cond 0x8583c40 was first observed at: ==7737== at 0x4C27C59: pthread_cond_init@* (drd_pthread_intercepts.c:652) ==7737== by 0x417573: boost::asio::detail::posix_event::posix_event() (in /tmp/a.out) ==7737== by 0x419F07: boost::asio::detail::task_io_service<boost::asio::detail::epoll_reactor>::idle_thread_info::idle_thread_info() (in /tmp/a.out) ==7737== by 0x419FBC: boost::asio::detail::task_io_service<boost::asio::detail::epoll_reactor>::run(boost::system::error_code&) (in /tmp/a.out) ==7737== by 0x417D26: boost::asio::io_service::run() (in /tmp/a.out) ==7737== by 0x430358: unsigned long boost::_mfi::mf0<unsigned long, boost::asio::io_service>::call<boost::shared_ptr<boost::asio::io_service> >(boost::shared_ptr<boost::asio::io_service>&, void const*) const (in /tmp/a.out) ==7737== by 0x42FF74: unsigned long boost::_mfi::mf0<unsigned long, boost::asio::io_service>::operator()<boost::shared_ptr<boost::asio::io_service> >(boost::shared_ptr<boost::asio::io_service>&) const (in /tmp/a.out) ==7737== by 0x42FE1E: unsigned long boost::_bi::list1<boost::_bi::value<boost::shared_ptr<boost::asio::io_service> > >::operator()<unsigned long, boost::_mfi::mf0<unsigned long, boost::asio::io_service>, boost::_bi::list0>(boost::_bi::type<unsigned long>, boost::_mfi::mf0<unsigned long, boost::asio::io_service>&, boost::_bi::list0&, long) (in /tmp/a.out) ==7737== by 0x42FD66: boost::_bi::bind_t<unsigned long, boost::_mfi::mf0<unsigned long, boost::asio::io_service>, boost::_bi::list1<boost::_bi::value<boost::shared_ptr<boost::asio::io_service> > > >::operator()() (in /tmp/a.out) ==7737== by 0x42EE79: boost::detail::thread_data<boost::_bi::bind_t<unsigned long, boost::_mfi::mf0<unsigned long, boost::asio::io_service>, boost::_bi::list1<boost::_bi::value<boost::shared_ptr<boost::asio::io_service> > > > >::run() (in /tmp/a.out) ==7737== by 0x4E41F5F: thread_proxy (in /usr/local/lib/libboost_thread.so.1.43.0) ==7737== by 0x4C2EA60: vgDrd_thread_wrapper (drd_pthread_intercepts.c:272) ==7737== mutex 0x61820f8 was first observed at: ==7737== at 0x4C2EC6B: pthread_mutex_init (drd_pthread_intercepts.c:537) ==7737== by 0x40AB63: boost::asio::detail::posix_mutex::posix_mutex() (in /tmp/a.out) ==7737== by 0x410B72: boost::asio::detail::task_io_service<boost::asio::detail::epoll_reactor>::task_io_service(boost::asio::io_service&) (in /tmp/a.out) ==7737== by 0x40F679: boost::asio::io_service::service* boost::asio::detail::service_registry::create<boost::asio::detail::task_io_service<boost::asio::detail::epoll_reactor> >(boost::asio::io_service&) (in /tmp/a.out) ==7737== by 0x40ADB8: boost::asio::detail::service_registry::do_use_service(boost::asio::io_service::service::key const&, boost::asio::io_service::service* (*)(boost::asio::io_service&)) (in /tmp/a.out) ==7737== by 0x40DDDE: boost::asio::detail::task_io_service<boost::asio::detail::epoll_reactor>& boost::asio::detail::service_registry::use_service<boost::asio::detail::task_io_service<boost::asio::detail::epoll_reactor> >() (in /tmp/a.out) ==7737== by 0x417C48: boost::asio::io_service::io_service() (in /tmp/a.out) ==7737== by 0x416921: http::server2::io_service_pool::io_service_pool(unsigned long) (in /tmp/a.out) ==7737== by 0x437816: http::server2::server::server(std::string const&, std::string const&, std::string const&, unsigned long) (in /tmp/a.out) ==7737== by 0x43064D: main (in /tmp/a.out) ==7737== ^C==7737== ==7737== For counts of detected and suppressed errors, rerun with: -v ==7737== ERROR SUMMARY: 3 errors from 2 contexts (suppressed: 225 from 194)

1 1

[endian] views and conversions
by vicente.botet 08 Jun '10

08 Jun '10

Hi, I have pushed the endian typed approach a little more. Given an C/C++ builtin integer, we can see it as an endian with a given endianness using a cast or view operation. This operation doesn't do any conversion when its parameter is a fundamental (integer) type, but just let the user states that the integer stored in the variable has been stored with the given endianness. int b; Let me say that b contains an integer with big endian format (this has been done by reading from an external device, e.g.) The following expression as<big>(i) or endian_cast<big>(i) will see the i storage as a big endian integer holder. Now we can convert from big to native as follows int n; as<native>(n) = as<big>(b); as<endianness>(x) can be implemented as a function that return a endian reference to an integer with a specific endianness. Let me call this class endian_view. endian_view will define the assignment operator so the there is a conversion when both sides have the same endianness. In place conversion can be done as follows: as<native>(b) = as<big>(b); In order to support the in place version the assignement operator on endian_views, needs to check in addition if the reference integer is the same. But this check could be inefficient when we are converting a whole structure. The same operation can be overloaded for endian_views and do a in_place conversion in this case as<native>(as<big>(b)); can be read as: make the integer b (which is formatted with big endian format) have a native endian format. Just some more thougths that can grow on the endian forest. Best, _____________________ Vicente Juan Botet Escribá http://viboes.blogspot.com/

1 0

[msm] Nested submachines and explicit state entry
by Christophe Henry 08 Jun '10

08 Jun '10

Hi Michael, >As already stated in the documentation it is possible to have explicit >state entry into another submachine within a state machine. > >Is there a possibility to have a state machine, which has another >submachine with explicit state entry and that submachine has another >submachine with an initial transition. >I put that together in a picture and attached it. I hope then it gets >clearer what I want to do. >The picture shows just a simplified version of the overall state machine >and I discarded some other states, transitions and orthogonal regions. Starts becoming an interesting case ;-) Yes it should work. I tested with the same depth-level without any problem. What is not supported yet would be if your explicit entry was going into a substate of a submachine of a submachine, meaning you can only "cross" one submachine boundary. So your example is ok. >How do I have to declare the transition table and states in that case? A transition table is always declared inside a state machine, after the states have been declared. In my examples I always declare states inside the front-end but it is only for readability. In your case, you could declare the SM1 front-end, then inside its declaration add the one for RunningStateMachine, then inside RunningStateMachine the one for Inner1 and Inner2. After Inner1 and Inner2, the table for RunningStateMachine, then the table for SM1. something like: struct SM1_... { struct RunningStateMachine_... { struct Inner1_...; typedef msm::back::state_machine<Inner1_> Inner1; ... // add RunningStateMachine table }; typedef msm::back::state_machine<RunningStateMachine_> RunningStateMachine; // add SM1 table }; typedef msm::back::state_machine<SM1_> SM1; >Are there any restrictions regarding nested depth? Not in the library. I see just 2: - your patience while compiling - your compiler. At some point, it will probably get pissed off ;-) Regards, Christophe

2 2

Bug sprint daily update for Tuesday, June 8th
by Marshall Clow 08 Jun '10

08 Jun '10

As of 0800 Pacific time today, we had 938 open trac tickets. That is down by 20 from yesterday, and 51 since the start of the bug sprint. Total ticket counts: Saturday 989 Sunday 988 Monday 958 Tuesday 938 Great work, everyone! Activity summary for yesterday: New tickets: 4 Modified: 31 Closed: 27 Reopened: 2 Today, rather than the summary totals for yesterday, I have deltas; changes in the totals. If you'ld like to see the totals, they are available online at: Open ticket count by owner: https://svn.boost.org/trac/boost/report/18 Open ticket count by component: https://svn.boost.org/trac/boost/report/19 Open ticket count by type https://svn.boost.org/trac/boost/report/20 (Overwhelmingly, these are 'bugs') Open ticket count by milestone: https://svn.boost.org/trac/boost/report/21 (All changes are since the start of the bug sprint; 0800 PDT Saturday June 5th) Changed totals by library: None -17 thread -16 website -10 units -3 Building Boost -2 random -2 GIL -1 Regression Testing -1 asio -1 date_time -1 filesystem -1 function -1 fusion -1 iterator -1 lexical_cast -1 mpi -1 mpl -1 regex -1 spirit -1 circular_buffer 1 foreach 1 program_options 1 serialization 1 utility 1 Documentation 2 graph 2 build 4 Changed totals by author: <none> -20 anthonyw -15 grafik -9 dave -2 dgregor -2 matthiasschabel -2 agurtovoy -1 az_sw_dude -1 bemandawes -1 bschaeling -1 chris_kohlhoff -1 danieljames -1 djowel -1 dlwalker -1 hkaiser -1 hljin -1 johnmaddock -1 nasonov -1 no-maintainer -1 speedsnail -1 steven_watanabe -1 danielw 1 eric_niebler 1 jano_gaspar 1 matias 1 ramey 1 witt 1 asutton 2 vladimir_prus 7 Changed totals by category: Bugs -37 Feature Requests -7 Patches -3 Tasks -3 -- Marshall

1 0

Re: [boost] Review of a safer memory management approach for C++
by Bartlett, Roscoe A 08 Jun '10

08 Jun '10

Mathias, > ------------------------------ > > Message: 19 > Date: Mon, 07 Jun 2010 15:43:18 +0100 > From: Mathias Gaunard <mathias.gaunard(a)ens-lyon.org> > To: boost(a)lists.boost.org > Subject: Re: [boost] Review of a safer memory management approach for > C++? > Message-ID: <huj0io$6iu$1(a)dough.gmane.org> > Content-Type: text/plain; charset=ISO-8859-1; format=flowed > > Bartlett, Roscoe A wrote: > > > I am fine with the language allowing for undefined behavior. Clearly > if you want the highest performance, you have to turn off array-bounds > checking, for instance, which allows for undefined behavior. What I > am not okay with is people writing programs that expose that undefined > behavior, especially w.r.t. to usage of memory. Every computational > scientist has had the experience of writing code that appeared to work > just fine on their main development platform but when they took to over > to another machine for a "production" run on a large (expensive) MPP to > run on 1000 processors, it segfaulted after having run for 45 minutes > and lost everything. This happens all the time with current CSE > software written in C, C++, and even Fortran. This is bad on many > levels. > > > > Are you suggesting that people should write programs that rely on > undefined behavior or are you just saying that you don't think it > should be eliminated from the language? > > From my experience, the best tool to identify and eliminate undefined > behaviour due to bad memory access is valgrind. > > I found it more practical and catching more errors than any > library-level tool. [Bartlett, Roscoe A] Tools like valgrind and purify are very helpful but are not nearly sufficient as described in Section 3.2 (and other sections) in: http://www.cs.sandia.gov/~rabartl/TeuchosMemoryManagementSAND.pdf Have other people had similar experiences with valgrind and purify to what I describe in Sectioin 3.2 in the above document? If valgrind and purify can pin-point memory errors so effectively, then why does it sometimes take nearly a week to track down certain types of memory errors in C++ (I have heard of such stories)? My record is most likely about three days to track down a single memory error (including using valgrind and purify for all the glues that I can get, less they segfaulted themselves of course which sometimes they do). In the process of tracking down and diagnosing some of these memory errors, I have had to apply all of my PhD-level problem solving and research skills and in the end I could not write a research paper to describe the wonderful problem-solving research-quality work that I did. What a waste of time and effort; very frustrating. I know many people that I work with that have had similar experiences. These types of experiences have lead many C++ teams to code C++ with a (justified but unfortunate) paranoia about the use of memory in C++ that has all kinds of bad consequences (see Section 1 in the above document for some of the consequences). Come one, at least some people on this mail list must have had similar nightmare experiences in trying to track down and diagnose hard memory misuse errors in C++ that took days of effort to resolve (even with the help of tools like valgrind and purify). And again, tools like valgrind and purify will *never* catch semantic misuse of memory (i.e. allocating a big chunk of memory and then breaking it up to be used to construct different objects and array of objects). The Teuchos MM classes will catch most semantic misuse of memory in a way that no tool like valgrind and purify every can (because they don't know the context of your program, they only know that any read/writes in a big block of memory that you allocated look okay). I think this is a big deal in catching hard-to-find defects that are not (technically speaking) memory misuse defects but are program defects none the less. Cheers, - Ross

6 9

[msm] Nested submachines and explicit state entry
by Michael.Herchel＠rohde-schwarz.com 08 Jun '10

08 Jun '10

Hi, As already stated in the documentation it is possible to have explicit state entry into another submachine within a state machine. Is there a possibility to have a state machine, which has another submachine with explicit state entry and that submachine has another submachine with an initial transition. I put that together in a picture and attached it. I hope then it gets clearer what I want to do. The picture shows just a simplified version of the overall state machine and I discarded some other states, transitions and orthogonal regions. How do I have to declare the transition table and states in that case? Are there any restrictions regarding nested depth? Best Regards, Michael

1 0

boost::thread compiler error on SUN CC 5.9
by Urs Stotz 08 Jun '10

08 Jun '10

boost::thread give a compile error on SUN CC 5.9: /opt/SUNWspro/bin/CC +d -library=stlport4 -library=stlport4 -xcode=pic32 -erroff=wvarhidemem,hidevf,hidevfinvb -errtags=yes -features=tmplife -features=tmplrefstatic -g -mt -erroff=%none -KPIC -DBOOST_ALL_NO_LIB=1 -DBOOST_THREAD_BUILD_DLL=1 -DBOOST_THREAD_POSIX -I"." -c -o "bin.v2/libs/thread/build/sun-5.9/debug/stdlib-sun-stlport/threading-multi/pthread/thread.o" "libs/thread/src/pthread/thread.cpp" "libs/thread/src/pthread/thread.cpp", line 77: Error, badbinaryop: The operation "boost::shared_ptr<boost::detail::tss_cleanup_function> && void*" is illegal. "libs/thread/src/pthread/thread.cpp", line 588: Error, badbinaryop: The operation "boost::shared_ptr<boost::detail::tss_cleanup_function> || void*" is illegal. 2 Error(s) detected. make: *** [thread.o] Error 2 with this small patch it will compile: --- boost_1_42_0.org/libs/thread/src/pthread/thread.cpp 2009-10-19 11:18:13.000000000 +0200 +++ boost_1_42_0/libs/thread/src/pthread/thread.cpp 2010-02-10 13:48:25.037748000 +0100 @@ -74,7 +74,7 @@ namespace boost { current=next; ++next; - if(current->second.func && current->second.value) + if(current->second.func && (current->second.value != 0)) { (*current->second.func)(current->second.value); } @@ -585,7 +585,7 @@ namespace boost { (*current_node->func)(current_node->value); } - if(func || tss_data) + if(func || (tss_data != 0) ) { current_node->func=func; current_node->value=tss_data;

3 2

SPAM in trac
by Steven Watanabe 08 Jun '10

08 Jun '10

AMDG I just saw some spam here: https://svn.boost.org/trac/boost/ticket/1338 In Christ, Steven Watanabe

3 3

asio loopback behavior, windows vs. linux
by Adam Crain 08 Jun '10

08 Jun '10

There’s an interesting behavioral difference between how the loopback works with ASIO on windows/linux. I have constructed a test that talks to an echo server across the loopback. The test steps are: 1) Open the socket using run_one(), verifying it is open 2) Start an async read 3) Start an async write 4) Call run_one, then verify the write succeeds 5) Call run_one, then verify that the data you write is received 6) Goto step 2 some number of times since the behavior is non-determinsitic One windows, this seems to consistently fail within the first 100 iterations, because on occasional you get the read callback with echoed data before the write success comes back!!!! This seems crazy that another process answers you before the OS/ASIO has even told you the write was successful. On Linux I have not been able to make this happen even with 10K’s of iterations. I can only assume that: 1) This can actually happen with Windows loopback + thread scheduling and my assumption that write callbacks will always occur before response data is invalid 2) There is a bug in the windows impl of ASIO that makes this possible. If anyone can shed some light on this I’d really appreciate it. I have code that wants to step through a series of states, and it makes things a lot more complicated if I can’t assume that the write callback will always occur before data received that’s a response to what I sent. Thanks! Adam Adam Crain Manager of R&D Green Energy Corp. 919-428-1002

2 1

Re: [boost] [bjam][msm] help building tests
by Christophe Henry 07 Jun '10

07 Jun '10

>> I am almost ready for release, except that I run into a (admittedly >> expected) little snag. I have in the trunk a libs/msm/test. I also >> have a jamfile, which even compiles. But doesn't link. >> The reason being that bjam does not find the Boost.Test libs. Ok I >> understand I should be telling bjam to do it, but looking at other >> libraries' jamfiles, I don't find it either, so I'm wondering how this >> is supposed to work. Am I missing something? > > You can change libs/msm/test/Jamfile.v2 like this > > project msm > : > requirements > <include>. > <library>/boost/test//boost_unit_test_framework/<link>static > ; > > to link against Boost.Test libraries. I'm not sure but I think that's > all you need. > >> On a related note, I'm also wondering how to get the tests to show up >> on the trunk regression page. > > Look in status/Jamfile.v2 and follow the pattern (add line for msm/ > test in the appropriate place) to get your product's tests running. > > Hope that helps. > > -- Noel Hi Steven, Noel, Jürgen, Vicente, This is what I call getting good help fast! :) Thanks to all, I just did the changes and committed. Tests ran successfully (well, one 1 compiler...) and status/Jamfile.v2 now contains msm, so I eagerly wait for the results ;-) So I now go further through the check-list. From what I saw, the inspection report now considers msm as one of the worst offenders. Can't have that. ^^ Big big thanks to all! Christophe

1 0

[bjam][msm] help building tests
by Christophe Henry 07 Jun '10

07 Jun '10

Hi, I am almost ready for release, except that I run into a (admittedly expected) little snag. I have in the trunk a libs/msm/test. I also have a jamfile, which even compiles. But doesn't link. The reason being that bjam does not find the Boost.Test libs. Ok I understand I should be telling bjam to do it, but looking at other libraries' jamfiles, I don't find it either, so I'm wondering how this is supposed to work. Am I missing something? On a related note, I'm also wondering how to get the tests to show up on the trunk regression page. Thanks, Christophe

5 4

Re: [boost] [regex, xpressive] interesting(?) perf benchmark
by Christophe Henry 07 Jun '10

07 Jun '10

>> How about a warning "you may not be using the optimal tool for the job" >> until then? > > I'm having a hard time getting excited about that. :) >> Are the finite state machine libraries in Boost usable for this at all? > > I don't know. At BoostCon, Christophe presented some interesting results > pitting MSM against xpressive for some simple regexing. An interesting > research project would be to investigate whether xpressive's hybrid > static/dynamic NFA design can be applied to DFAs as well, with similar > perf wins. This would be a non-trivial undertaking, like completely > reimplementing xpressive. I think MSM could be a possibility, and it seems to be quite fast for some cases. I was actually thinking of handling static Xpressive as a MSM-compatible front-end, so that the user could switch at will between Xpressive and MSM, which happen to be complementary because most performant on different types of regexes. But there is a long way (and a fair number of technical issues) between a toy example and a real library so it could take a while before I get to it (besides, I am not a domain expert). Note: there is an example provided with the BoostCon slides. I have more and I can pack them with MSM's examples. Christophe

1 0

identifying file system type
by Roger Martin 07 Jun '10

07 Jun '10

Hi, I'm looking for a way code can automatically decide whether it is running on a parallel or non-parallel file system. Need to retrieve the information about the file system type. Get values like "ext3", "nfs" "gpfs", "pvfs", "Lustre" etc. Is there any file system info obtainable in code?

1 0

[trac] "Building Boost" vs. "build"
by Vladimir Prus 07 Jun '10

07 Jun '10

It looks like somebody went via Trac and moved some issues from the "Building Boost" component to "build" component. See, for example: https://svn.boost.org/trac/boost/ticket/4147 I certainly appreciate help with triaging issues, but in general, such change seems wrong -- "Building Boost" is for issues affecting exactly that -- building boost, while the 'build' component contain issues that are not necessary affecting Boost C++ Libraries at all. Unfortunately, the changes are all anonymous, so it would be great if whoever made them speak up, and discuss the rationale. Thanks, -- Vladimir Prus CodeSourcery vladimir(a)codesourcery.com (650) 331-3385 x722

4 5

Active Tickets report and accepted info
by vicente.botet 07 Jun '10

07 Jun '10

Hi, in https://svn.boost.org/trac/boost/report/1 I see the following: "If a ticket has been accepted, a '*' is appended after the owner's name" Is there an error on the report or the maintainers do not use to accept the tickets? I find this information very useful, if it states that the maintainer accept to take care of this ticket. Best, _____________________ Vicente Juan Botet Escribá http://viboes.blogspot.com/

5 7

Re: [boost] Review of a safer memory management approach for C++?
by Bartlett, Roscoe A 07 Jun '10

07 Jun '10

David, Much more discussion of this topic over email is likely not very useful at this point but I will make some very brief responses below. > 14. Re: FW: Boost Digest, Vol 2929, Issue 4 (David Abrahams) > ------------------------------ > > Message: 14 > Date: Fri, 04 Jun 2010 09:42:36 -0400 > From: David Abrahams <dave(a)boostpro.com> > To: boost(a)lists.boost.org > Subject: Re: [boost] FW: Boost Digest, Vol 2929, Issue 4 > Message-ID: <m2hbliewxf.wl%dave(a)boostpro.com> > Content-Type: text/plain; charset=UTF-8 > > At Thu, 3 Jun 2010 17:29:15 -0600, > Bartlett, Roscoe A wrote: > > > > David, > > > > I have some responses below ... > > > > > Message: 13 > > > Date: Tue, 01 Jun 2010 16:16:25 -0400 > > > From: David Abrahams <dave(a)boostpro.com> > > > > > > I'm all for making essential complexity obvious and eliminating > > > accidental complexity, but I'm not sure I see a compelling example > > > that Teuchos does that, at least not when compared with what I > > > consider normal state-of-the-art C++ programming, which is mostly > > > value-based, dynamic allocations are rare, and those that occur are > > > immediately managed, e.g. by appropriate smart pointers. > > > > [Bartlett, Roscoe A] > > > > I don't want to get into an argument about the "proper" way to use > > C++ here (but I will a little below). In Item 1 of "Effective C++, > > 3rd Edition", Scott Meyers identifies four different major > > programming paradigms that are supposed to be supported by C++: > > > > 1) C: raw pointers, low-level built-in datatypes (int, double, > > etc.), statements, blocks, functions etc. > > > > 2) Object-oriented C++: Inheritance, virtual functions, dynamic > > allocation, RTTI, etc. (i.e. runtime polymorphism) > > > > 3) Template C++: Typical generic programming, template > > metaprogramming, etc. (i.e. compile-time polymorphism) > > > > 4) STL: Containers, iterators, algorithms, etc. and code like this > > (not just using STL) (closely related to #3 Template C++). > > Yes, I'm aware of that way of dividing up the picture. > > > Of course any well written large C++ program is a mixture of all > > four of the above (and more). > > I don't take it for granted that any well-written large C++ program > will use any given paradigm. > > > What you seem to be arguing above is that most good C++ software > > will use templates in #3 and #4 but almost none of #2 > > object-oriented C++. > > Did I say anything about templates? No. > > I _will_ argue that OOP has costs to software complexity that are > often under-appreciated. [Bartlett, Roscoe A] Any approach can be misused, including anything you might be trying to advocate (but I am not 100% sure what that is exactly). > > Note that C++ was first designed in the early > > 80s primarily to support #2 object-oriented C++. I can see the > > reason for this bias for #3 and #4 given the nature of most of the > > Boost libraries and you can have that opinion if you would like. > > I don't, so please read what I wrote and try to address that rather > than to your preconceived notions of my ?bias.? [Bartlett, Roscoe A] I am not trying to put words in your mouth. I am just trying to pin down your argument/position. You can say that I have a "bias" for good software if you would like :-) Seriously, I am sorry if I misinterpreted you position with the position of others on the boost list who were arguing for a more template-heavy approach. I guess that you are arguing for more of a stack-based approach (see below). > > I would argue that the current accepted approaches to writing OO > > programs in C++ (i.e. #2 OO C++ features) make it too easy to create > > programs with undefined behavior (i.e. segfaults or worse) and as a > > result, the paranoia of undefined behavior in C++, memory leaks > > etc. lead to lots of bad programming practices and designs (see > > Section 1 in > > http://www.cs.sandia.gov/~rabartl/TeuchosMemoryManagementSAND.pdf) > > Seriously? Are you saying that a disciplined application of > shared_ptr, weak_ptr, new_shared, et. al when dynamic allocations are > called for is insufficient for most OO programs? [Bartlett, Roscoe A] As argued in the Teuchos MM document http://www.cs.sandia.gov/~rabartl/TeuchosMemoryManagementSAND.pdf these boost classes are a good start but they are *not* sufficient to build a safe and effective memory management system that encapsulated *all* raw pointers. What is missing for single objects, as described in Section 5.14 and other sections, is the Teuchos::Ptr class. Without integrating a class like Ptr in with the reference-counted pointer classes, you can't build a safe and effective memory management system in C++ that comes closer to eliminating undefined behavior. Also, as described in Section 5.9.2, separating weak_ptr from shared_ptr is not ideal from a design perspective since it reduces the generality and reusability of software (see the arguments for this in the document). Did you read at least the sections called out in the reduced table of contents given in the Preface? I think these sections should address all of these questions (and more). As for arrays, the current software in C++0x and boost is less sufficient (again, see Section 5.14 and the sections it refers to). What is in boost and C++0x is a good start but it has not gone far enough (i.e. the encapsulation of *all* raw pointers in *all* code, except where non-compliant code must be called of course). > > > > Trying to over design a program to avoid all shared ownership is > > > > what make C++ programming so unproductive and has all the > negative > > > > consequences described in Section 1 in: > > > > > > > > > http://www.cs.sandia.gov/~rabartl/TeuchosMemoryManagementSAND.pdf > > > > > > > > Designs with object sharing can be much less complex overall than > > > > designs without sharing. You just need decent tools to detect > circular > > > > reference problems (and that is what the Teuchos::RCP class has). > > > > > > Well, I fundamentally disagree with all of the above. Overuse of > > > runtime polymorphism, thus dynamic allocation, and thus shared > > > ownership (it's almost an inevitable progression) is one of the > things > > > that has made C++ programming unproductive, and those people I know > > > who most zealously avoid it tend to be more productive than > everyone > > > else. IMO. > > > > [Bartlett, Roscoe A] > > > > When runtime performance or other issues related to dynamic > > allocations and classic OO are not a problem, classic OO C++ > > programs using runtime polymorphism are typically superior to highly > > templated C++ programs using static polymorphism > > Straw man; I'm not arguing for ?templated C++ programs using static > polymorphism.? [Bartlett, Roscoe A] Okay, the runtime vs. compile-time polymorphism debate is another issue (but a very important one). Are you instead arguing for a static and stack-based approach to programming? I haven see people try this and it leads to very inflexible software that is hard to maintain, reuse, and achieve optimal performance. One particularly good (or bad) example of this approach that I have seen is given in: http://www.trip.caam.rice.edu/software/rvldoc/rvl.html If you look deep in the RVL code, you will see a few Gotchas that destroy the true safely that is supposed to be afforded by the approach. Actually, ironically, they use stack-based object to try to avoid memory management problems (but not sufficiently) but yet at the same time, they rely on other types of mutable shared objects with implicit updates that make my very unconformable. > > > In my experience, designs with object sharing tend to increase > > > complexity in all kinds of ways beyond the memory management issues > > > that you describe. Perhaps the most important way is, they tend to > > > obscure the overall computation being performed by hiding it in > > > networks of interacting objects. > > > > [Bartlett, Roscoe A] > > > > This is not going to magically change by templating everything. > > Straw man; I'm not arguing for ?templating everything.? > > > If a lot of objects collaborate together to perform computations, it > > does not matter if you use static or runtime polymorphism; the code > > will be hard to figure out and you will need print statements and/or > > run it in a debugger to see what is really happening. > > Yes, and an ?OO mindset? has tended to lead to partitioning systems so > that ?a lot of objects collaborate together to perform computations,? > which tends to obscure the computation performed. [Bartlett, Roscoe A] Yes but good OO design tries to minimize the web of object dependences. There are lots of good references on the need and methods for reducing the endless web of object dependencies (see http://www.cs.sandia.gov/~rabartl/readingList.html) > > > I would like to see an example of a design with shared object > > > ownership that is ?much less complex? than all other functionally > > > equivalent designs that eschew shared ownership. I'm not saying > such > > > applications don't exist, but IME they are rare, and a concrete > > > example would help to bring this whole discussion back to earth. > > > > [Bartlett, Roscoe A] > > > > One design where shared mutable objects likely makes things overall > > simpler and more natural is the PDE expression mechanism in the > > Sundance code (http://www.math.ttu.edu/~klong/Sundance/html/) If > > the clients all expect remote updates of shared objects, then things > > work great. > > I'm sorry, your reference isn't specific enough for me to see the > design piece you're referring to. [Bartlett, Roscoe A] It would take a significant effort to pin-point exactly the approach being used in the Sundance expression mechanism and then to describe an alternative design that does not rely on the kind of object sharing. I think that this example in Sundance is the exception and not the rule. In general, I don't like lots for mutable shared objects for the same reasons that you state. However, there are cases where using shared mutable objects is the best design to solve a given problem. As for dynamic vs. stack-based programming, in summary, if you like the approach used in: http://www.trip.caam.rice.edu/software/rvldoc/rvl.html which tries to to make *every* object a stack-based object, then you and I can safely agree to disagree about classic OO vs. purely stack-based programming. Sorry I misinterpreted your position (or projected other peoples positions on you) about the runtime vs. compile-time polymorphism debate. Cheers, - Ross

4 7

FW: Boost Digest, Vol 2929, Issue 4
by Bartlett, Roscoe A 07 Jun '10

07 Jun '10

David, I have some responses below ... > ------------------------------ > > Message: 13 > Date: Tue, 01 Jun 2010 16:16:25 -0400 > From: David Abrahams <dave(a)boostpro.com> > To: boost(a)lists.boost.org > Cc: "mathias.gaunard(a)ens-lyon.org" <mathias.gaunard(a)ens-lyon.org> > Subject: Re: [boost] Review of a safer memory management approach for > C++? > Message-ID: <m2hblmcxuu.wl%dave(a)boostpro.com> > Content-Type: text/plain; charset=UTF-8 > > ... > > > > At Thu, 27 May 2010 20:02:45 +0200, > > > Ingo Loehken wrote: > > > > > > > > if I understand "hard to reason about" in the right why : like > there > > > > is no need for shared ownership at all, this also means that > there > > > > is no use for COM Programming - and of course there is. > > > > > > I think you don't understand it the right way. Shared ownership > (at > > > least in the presence of mutation) is hard to reason about because > > > seemingly-local modifications can have non-local effects. > > > > [Bartlett, Roscoe A] > > > > Yes, but let's focus on the "essential complexity" of object sharing > > Okay. > > > and not get bogged down in the "accidental complexity" of memory > > management. The approaches described in > > > > http://www.cs.sandia.gov/~rabartl/TeuchosMemoryManagementSAND.pdf > > > > make explicit the "essential complexity" of object sharing while > > trying to eliminate the "accidental complexity" of memory management > > (see Section 6.1 in > > http://www.cs.sandia.gov/~rabartl/TeuchosMemoryManagementSAND.pdf) > > I'm all for making essential complexity obvious and eliminating > accidental complexity, but I'm not sure I see a compelling example > that Teuchos does that, at least not when compared with what I > consider normal state-of-the-art C++ programming, which is mostly > value-based, dynamic allocations are rare, and those that occur are > immediately managed, e.g. by appropriate smart pointers. [Bartlett, Roscoe A] I don't want to get into an argument about the "proper" way to use C++ here (but I will a little below). In Item 1 of "Effective C++, 3rd Edition", Scott Meyers identifies four different major programming paradigms that are supposed to be supported by C++: 1) C: raw pointers, low-level built-in datatypes (int, double, etc.), statements, blocks, functions etc. 2) Object-oriented C++: Inheritance, virtual functions, dynamic allocation, RTTI, etc. (i.e. runtime polymorphism) 3) Template C++: Typical generic programming, template metaprogramming, etc. (i.e. compile-time polymorphism) 4) STL: Containers, iterators, algorithms, etc. and code like this (not just using STL) (closely related to #3 Template C++). Of course any well written large C++ program is a mixture of all four of the above (and more). What you seem to be arguing above is that most good C++ software will use templates in #3 and #4 but almost none of #2 object-oriented C++. Note that C++ was first designed in the early 80s primarily to support #2 object-oriented C++. I can see the reason for this bias for #3 and #4 given the nature of most of the Boost libraries and you can have that opinion if you would like. However, there are clearly types of programming problems that call for more of #2 and less of #3 and #4 in the overall design. Languages like Java and Python would be typically used to write safe object-oriented programs in a productive environment (free of complications of dynamic memory allocation and most undefined behavior that is so easy to expose with current C++ approaches). I will argue below that more C++ programs are better served by using more of #2 object-oriented C++ and to a lesser extent template-based methods in #3 and #4 (except where it is called for in lower-level code). Instead of arguing if we should be using #2 object-oriented C++ (runtime polymorphism) or instead prefer #3 template programming (static polymorphism), let's assume that we want C++ to do a good job of supporting #2 object-oriented C++ if we decide to use it. Therefore, the real question is if C++ is indeed supposed to support #2 object-oriented programming, does C++ it support it well enough so that I would choose to use C++ instead of Java or Python (assuming my programming team knows those languages)? By all reasonable measures, writing object-oriented programs in Java or Python is more productive because of garbage collection and the elimination of undefined behavior that is so common in C++ programs (I can find lots of references for these opinions and even data from studies to support this claim). However, to achieve performance in CSE software, I have to write some code in C/C++ or some other appropriate language. No-one has been able to demonstrate sufficient performance for CSE codes written entirely in Java or Python. Also, many of the "safer" and "more productive" OO languages (e.g. Java and Python) are not even available on some high-end HPC platforms. Even if they were, mixed language programming has lots of problems that discourages wide-spread use. That leaves C++ as the only fully portable viable language for high-performance CSE (sorry Fortran programmers). I would argue that the current accepted approaches to writing OO programs in C++ (i.e. #2 OO C++ features) make it too easy to create programs with undefined behavior (i.e. segfaults or worse) and as a result, the paranoia of undefined behavior in C++, memory leaks etc. lead to lots of bad programming practices and designs (see Section 1 in http://www.cs.sandia.gov/~rabartl/TeuchosMemoryManagementSAND.pdf) What the approach described in the Teuchos MM report tries to do is to create a set of classes and idioms that can allow C++ to be used to safely and productively develop #2 OO C++ software closer to the safety and productively that you can get with languages like Java and Python. However, there is still too much "accidental complexity" in C++ to have it ever fully compete with productivity of writing in Java and Python for basic OO features (IMO). In any case, C++ offers other advantages over any other language that would have you choose it over these other languages. > > Trying to over design a program to avoid all shared ownership is > > what make C++ programming so unproductive and has all the negative > > consequences described in Section 1 in: > > > > http://www.cs.sandia.gov/~rabartl/TeuchosMemoryManagementSAND.pdf > > > > Designs with object sharing can be much less complex overall than > > designs without sharing. You just need decent tools to detect circular > > reference problems (and that is what the Teuchos::RCP class has). > > Well, I fundamentally disagree with all of the above. Overuse of > runtime polymorphism, thus dynamic allocation, and thus shared > ownership (it's almost an inevitable progression) is one of the things > that has made C++ programming unproductive, and those people I know > who most zealously avoid it tend to be more productive than everyone > else. IMO. [Bartlett, Roscoe A] When runtime performance or other issues related to dynamic allocations and classic OO are not a problem, classic OO C++ programs using runtime polymorphism are typically superior to highly templated C++ programs using static polymorphism for the following (indisputable) reasons: 1) Well written OO C++ software builds and (more importantly) rebuilds *much* faster then heavily template code. This massively speeds up the development cycle. The difference can be orders of magnitude of time savings in recompile/relink times. I have seen cases where carefully using runtime OO can result in recompile/relink cycles take less than 20 seconds where using all template code would take 20 minutes. I am *not* exaggerating here at all. 2) Templated programs can result in massive object-code bloat over equivalent runtime OO programs. If you are really sloppy with templates and use it for everything, you can easily double, triple, or more the size of the object code and executables (see Item 44 in "Effective C++, 3ed edition"). 3) The error messages you get when mistakes are made when using interfaces and virtual functions are far superior to the mess that most compliers put out when you make a mistake with implicit compile-time template interfaces. This is a huge issue. Runtime errors, as opposed to cryptic compile-time errors, can be handled by throwing exceptions with very good error messages that programmers can 100% control the quality of. Yes, of course you prefer compile-time errors to runtime errors all things being equal, and am not arguing with that. This is generally not true of template programming (but there are some tricks you can use to improve compile-time error messages, but not perfectly). For example, if you make a mistake with STL iterators/algorithms you can get very cryptic error messages that dumfound most (non-expert and expert) C++ programmers. 4) Well written OO C++ software allows for significant runtime flexibility. If you use dynamic libraries, for instance, you can add new implementations of objects without even having to relink customer executables. Also, you can fix many types of bugs in updated shared libraries without having to recompile/relink customer executables. In a large-scale software setting, this can be very helpful. Believe me, I see the use of template C++ code with lots of static typing and lots of local logic and control but it is not the end all and will never replace classic OO C++ for most high-level designs and software architecture. I challenge anyone to refute this. C++ compilers will need to become many orders of magnitude faster that what they are today for this fact to change (but then the software will just grow that much larger and erase the compiler speed improvements). > In my experience, designs with object sharing tend to increase > complexity in all kinds of ways beyond the memory management issues > that you describe. Perhaps the most important way is, they tend to > obscure the overall computation being performed by hiding it in > networks of interacting objects. [Bartlett, Roscoe A] This is not going to magically change by templating everything. If a lot of objects collaborate together to perform computations, it does not matter if you use static or runtime polymorphism; the code will be hard to figure out and you will need print statements and/or run it in a debugger to see what is really happening. > I would like to see an example of a design with shared object > ownership that is ?much less complex? than all other functionally > equivalent designs that eschew shared ownership. I'm not saying such > applications don't exist, but IME they are rare, and a concrete > example would help to bring this whole discussion back to earth. [Bartlett, Roscoe A] One design where shared mutable objects likely makes things overall simpler and more natural is the PDE expression mechanism in the Sundance code (http://www.math.ttu.edu/~klong/Sundance/html/) If the clients all expect remote updates of shared objects, then things work great. The problem is when an object is changed by another client and you don't expect it to change (i.e. the classic problem of change propagation). As for sharing of objects, it is critical for many types of software in the Computational Science and Engineering (CSE) domain. You have to share large objects in order to cut down on runtime and storage overhead. As one example, consider the Petra object model with the Trilinos packages Epetra and Tpetra (http://trilinos.sandia.gov/) In the Petra object model, you have objects with significant storage overhead like Maps and Graphs that are shared between many different objects like Vectors, MultiVectors, and Matrices. Yes, you could create deep copies of Maps and Graphs but that would waste a large amount of storage. The bottom line is that by sharing large objects like Maps and Graphs instead of making large numbers of copies, we can fit and solve much larger simulation problems on a computer than we could otherwise. You could template and statically allocate everything and you will never get around the fundamental need to share the data contained in large objects in these types of applications. Note, however, that Maps in Epetra/Tpetra are shared as const immutable objects. That means that no-one can change a Map after it is first created. Therefore, all of the problems with unexpected updates goes away (as you mention above). However, the shared Maps must go away when they are not needed anymore and that is what the Teuchos::RCP class makes easy and robust. Under the covers, Tpetra uses Tpetra::ArrayRCP for similar purposes and much more. The situation with Graphs is different and issues of change propagation of shared objects are still present in some cases. Come to think of it, most of the object sharing that used in the CSE software that I use and write mostly just share const immutable objects so problems of change propagation mostly goes away. However, there are some important use cases where all of the clients are not holding RCPs to const objects and the problem of change propagation remains. Again, to save in significant runtime and storage overhead, we can't just create deep copies of all of these objects. Templating and purely stack-based programs are not going to solve that problem. The Teuchos MM approach, I believe, largely solves the memory management problems with dynamic memory allocation and object sharing while still yielding very high performance and mostly eliminating undefined behavior associated with the incorrect usage of memory. The most significant contribution of the Teuchos MM approach is the presents of Teuchos::Ptr and Teuchos::ArrayView and how they interact with the other classes. Cheers, - Ross

6 9

How to retrieve original string back from hashed string using string_hash()?
by Tiwari, Sumit [Tech] 07 Jun '10

07 Jun '10

Hi, I can hash a string using string_hash(), is there any function available to get back the string value back by supplying hash value? Thanks, Sumit

2 1

Re: [boost] Review of a safer memory management approach for
by Bartlett, Roscoe A 07 Jun '10

07 Jun '10

Fernando and Jeffrey, A few responses ... > ------------------------------ > > Message: 6 > Date: Fri, 04 Jun 2010 00:14:25 -0300 > From: Fernando Cacciola <fernando.cacciola(a)gmail.com> > To: boost(a)lists.boost.org > Subject: Re: [boost] Review of a safer memory management approach for > C++? > Message-ID: <hu9r1a$57q$1(a)dough.gmane.org> > Content-Type: text/plain; charset=UTF-8; format=flowed > > ... > > For instance, it's been so long since I last wrote a virtual function > that > recently, I made the silly mistake of forgetting that, in a derived > class, an > overrided function doesn't have to specify the virtual keyword ;) [Bartlett, Roscoe A] High warning levels (i.e. with GCC) catch most such mistakes that the language proper allows to go through. BTW, are you gravitating to stack-based programming because of the memory management problems or because of problems of change propagation of shared mutable objects? These are almost 100% unrelated issues in my opinion. Is the C++ boost community really ready to throw away OO programming? That just seems crazy to me. > ------------------------------ > > Message: 9 > Date: Thu, 03 Jun 2010 21:12:52 -0700 > From: "Jeffrey Lee Hellrung, Jr." <jhellrung(a)ucla.edu> > To: boost(a)lists.boost.org > Subject: Re: [boost] FW: Boost Digest, Vol 2929, Issue 4 > Message-ID: <4C087D44.1080909(a)ucla.edu> > Content-Type: text/plain; charset=ISO-8859-1; format=flowed > > ... > > > As for sharing of objects, it is critical for many types of software > in the Computational Science and Engineering (CSE) domain. You have to > share large objects in order to cut down on runtime and storage > overhead. As one example, consider the Petra object model with the > Trilinos packages Epetra and Tpetra (http://trilinos.sandia.gov/) In > the Petra object model, you have objects with significant storage > overhead like Maps and Graphs that are shared between many different > objects like Vectors, MultiVectors, and Matrices. Yes, you could > create deep copies of Maps and Graphs but that would waste a large > amount of storage. The bottom line is that by sharing large objects > like Maps and Graphs instead of making large numbers of copies, we can > fit and solve much larger simulation problems on a computer than we > could otherwise. You could template and statically allocate everything > and you will never get around the fundamental need to share the data > contained in large objects > in > > these types of applications. > > > > Note, however, that Maps in Epetra/Tpetra are shared as const > immutable objects. That means that no-one can change a Map after it is > first created. Therefore, all of the problems with unexpected updates > goes away (as you mention above). However, the shared Maps must go > away when they are not needed anymore and that is what the Teuchos::RCP > class makes easy and robust. Under the covers, Tpetra uses > Tpetra::ArrayRCP for similar purposes and much more. The situation > with Graphs is different and issues of change propagation of shared > objects are still present in some cases. > > > > Come to think of it, most of the object sharing that used in the CSE > software that I use and write mostly just share const immutable objects > so problems of change propagation mostly goes away. However, there are > some important use cases where all of the clients are not holding RCPs > to const objects and the problem of change propagation remains. Again, > to save in significant runtime and storage overhead, we can't just > create deep copies of all of these objects. Templating and purely > stack-based programs are not going to solve that problem. > > > > The Teuchos MM approach, I believe, largely solves the memory > management problems with dynamic memory allocation and object sharing > while still yielding very high performance and mostly eliminating > undefined behavior associated with the incorrect usage of memory. The > most significant contribution of the Teuchos MM approach is the > presents of Teuchos::Ptr and Teuchos::ArrayView and how they interact > with the other classes. > > I don't see how shared object *ownership* is necessary in this > example... :/ I don't think Dave was arguing that holding large > objects > by reference, or even dynamically allocating objects (large or not), is > necessarily complex or "bad"... > > Or did I miss something? [Bartlett, Roscoe A] Have you read the key sections in the Teuchos MM approach in http://www.cs.sandia.gov/~rabartl/TeuchosMemoryManagementSAND.pdf or are you just responding to the list emails? You should *never* hold an object by reference as a class data-member (except perhaps in very well encapsulated very low-level code). That is what leads to undefined behavior with dangling references. Early versions of Epetra did just that and it blew up in people's faces. Consider writing a factory method such as: Epetra_CrsMatrix* createMyMatrix(....) { Epetra_Map rowMap(...); Epetra_Map colMap(...); ... Epetra_CrsGraph graph(rowMap, colMap); ... Epetra_CrsMatrix *matrix = new Epetra_CrsMatrix(graph); ... return matrix; } In early Versions of Epetra, maps and graphs were held and references and the above factory function returned an Epetra_CrsMatrix object with now deleted map and graph objects. Later, Epetra was refactored to make Epetra_Map and Epetra_Graph handle classes with internal reference-counted handle/body implementation objects. This fixed the problem but it create lots of new problems including that now copying graphs silently went from deep copy to shallow copy (which broke some code in some rare cases). The new Tpetra package has gone the down the road of making shared ownership explicit by exposing RCP-wrapped objects making factory functions similar to the above safe and obvious in behavior. It is working very well. Cheers, - Ross

8 8

Re: [boost] [endian] swap_in_place use case
by Dave Handley 07 Jun '10

07 Jun '10

Please don't drop attributions. Vicente Botet wrote: > Dave Handley wrote: >> Memory map a network endian file. Swap_in_place. Use. >> >> You definitely don't want a copy in this case since your file could easily be >> very big. Think of the case where your file is huge (say 10Gb), you really >> don't want to perform a copy and swap since that puts your memory need at 20Gb >> instead of 10Gb. > Yes this could be a use case. I don't use to manage so bigs files. If I had > to work with I will never copy the whole file. But I'm not sure that I will > use a swap_in_place of the whole file. This could take too much time. I would > try to split the task to do on the whole file in smaller parts. > > What will you do with this big file, that makes the swap in place the best choice? The file could be many things. It could be a day of market data for a given exchange. It could be image data or video data that I'm going to perform image analysis on (maybe run a filter over it, or something similar). The file doesn't even have to be that big. If I was memory mapping a 10MB file and needed to swap it, I wouldn't want to use 20MB instead of 10MB. Pretty much anything I want to do to that file that involves looking at most or all of the data you would be much better off using swap in place instead of any copying swap implementation. Examples of the sorts of things that you might want to do to large files include running filters or normalisers over image or video files. I have lots of programs that have multiple threads constantly memory mapping files that range in size from relatively small to hundreds of MB or low numbers of GB. Given that memory allocation is a key component of the run time of these programs, they would run significantly slower if I had to allocate double the amount of memory. Don't forget, if you need the whole file to be swapped, then the fastest way to do it will be a swap in place of the whole file. I will reiterate something I said in an earlier post. If boost accepts an endian library which does not provide an efficient swap in place, I will be unable to use it. The library will end up in the list of boost libraries which are too inefficient to use in performance sensitive production code. Dave Handley

5 6

The bug sprint has started!
by Marshall Clow 07 Jun '10

07 Jun '10

As of 0800 Pacific time today, we had 989 open trac tickets. Info about the bug sprint can be found at <https://svn.boost.org/trac/boost/wiki/BugSprintJune2009> By ticket type: Bugs 613 Feature Requests 238 Patches 95 Tasks 27 Support Requests 15 Library Submissions 1 By milestone: To Be Determined 187 Boost 1.43.0 180 Boost 1.42.0 132 Boost 1.40.0 85 Boost 1.36.0 82 Boost 1.41.0 81 Boost 1.39.0 67 <none> 61 Boost 1.38.0 45 Boost 1.37.0 37 Boost 1.44.0 20 Website 1.X 10 Boost.Jam 4.0.0 1 Top 10 libraries: thread 76 date_time 55 Python 48 test 43 interprocess 40 iostreams 37 filesystem 36 mpl 36 quickbook 33 asio 31 Top 10 owners: anthonyw 75 djowel 71 dave 58 igaztanaga 58 az_sw_dude 53 vladimir_prus 49 bemandawes 46 rogeeff 43 agurtovoy 36 turkanis 36 Useful reports: https://svn.boost.org/trac/boost/report/1 -- full list of tickets https://svn.boost.org/trac/boost/report/18 -- ticket counts by owner https://svn.boost.org/trac/boost/report/19 -- ticket counts by component https://svn.boost.org/trac/boost/report/20 -- ticket counts by ticket type https://svn.boost.org/trac/boost/report/21 -- ticket counts by milestone

3 3