[interprocess] Segmentation fault with 1.34+ on Linux x86_64
If I compile the attached file with a version of trunk current as of a few days ago it works fine on 32 bit Linux but produces a seg fault on 64 bit Linux. An older version of 1.34+ from a few months ago works fine. I configured using: ./configure --prefix=$HOME/boost_test --without-libraries=python,serialization,graph,test Has anybody else seen this behavior? Thanks, Glenn Schrader - MITLL
Glenn Schrader wrote:
If I compile the attached file with a version of trunk current as of a few days ago it works fine on 32 bit Linux but produces a seg fault on 64 bit Linux. An older version of 1.34+ from a few months ago works fine.
I configured using:
./configure --prefix=$HOME/boost_test --without-libraries=python,serialization,graph,test
Thanks for reporting it. I have no access to a 64 bit machine right noow but I'll try to get it ASAP. The example seems simple. I think you've already tested it, but are you sure it's a segfault and not an exception being thrown? If so, could you launch it with the debugger to see where the error is notified? Regards, Ion
Using gdb I get the following seg fault. I also included a backtrace. (gdb) run Starting program: /home/gschrad/projects/boost/test/shm_test [Thread debugging using libthread_db enabled] [New Thread 46912496320976 (LWP 9409)] Program received signal SIGSEGV, Segmentation fault. [Switching to Thread 46912496320976 (LWP 9409)] 0x000000000040202c in boost::interprocess::offset_ptr<boost::intrusive::detail::compact_rbtree_node<boost::interprocess::offset_ptr<void>
::get_pointer (this=0xaaaad058) at ../ins-1.35.PR1.2-linux-x86_64/include/boost-1_35/boost/interprocess/offset_ptr.hpp:80 /home/gschrad/projects/boost/ins-1.35.PR1.2-linux-x86_64/include/boost-1_35/boost/interprocess/offset_ptr.hpp:80:2541:beg:0x40202c (gdb) frame 0 #0 0x000000000040202c in boost::interprocess::offset_ptr<boost::intrusive::detail::compact_rbtree_node<boost::interprocess::offset_ptr<void> ::get_pointer (this=0xaaaad058) at ../ins-1.35.PR1.2-linux-x86_64/include/boost-1_35/boost/interprocess/offset_ptr.hpp:80 (gdb) frame 0 #0 0x000000000040202c in boost::interprocess::offset_ptr<boost::intrusive::detail::compact_rbtree_node<boost::interprocess::offset_ptr<void> ::get_pointer (this=0xaaaad058) at ../ins-1.35.PR1.2-linux-x86_64/include/boost-1_35/boost/interprocess/offset_ptr.hpp:80 (gdb) frame 0 #0 0x000000000040202c in boost::interprocess::offset_ptr<boost::intrusive::detail::compact_rbtree_node<boost::interprocess::offset_ptr<void> ::get_pointer (this=0xaaaad058) at ./ins-1.35.PR1.2-linux-x86_64/include/boost-1_35/boost/interprocess/offset_ptr.hpp:80
::get_pointer (this=0xaaaad058) at ../ins-1.35.PR1.2-linux-x86_64/include/boost-1_35/boost/interprocess/offset_ptr.hpp:80 #1 0x0000000000402075 in boost::interprocess::offset_ptr<boost::intrusive::detail::compact_rbtree_node<boost::interprocess::offset_ptr<void> ::get (this=0xaaaad058) at ../ins-1.35.PR1.2-linux-x86_64/include/boost-1_35/boost/interprocess/offset_ptr.hpp:141 #2 0x00000000004022ad in boost::intrusive::pointer_plus_bit<boost::interprocess::offset_ptr<boost::intrusive::detail::compact_rbtree_node<boost::interprocess::offset_ptr<void>
::get_bit (n=@0xaaaad058) at ../ins-1.35.PR1.2-linux-x86_64/include/boost-1_35/boost/interprocess/offset_ptr.hpp:393 #3 0x00000000004022d9 in boost::intrusive::detail::compact_rbtree_node_traits_impl<boost::interprocess::offset_ptr<void> ::get_color (n=@0x7fff4b460980) at ../ins-1.35.PR1.2-linux-x86_64/include/boost-1_35/boost/intrusive/detail/rbtree_node.hpp:144 #4 0x0000000000404f3f in boost::intrusive::rbtree_algorithms<boost::intrusive::detail::rbtree_node_traits<boost::interprocess::offset_ptr<void> ::rebalance (p=@0x7fff4b460db0, header=@0x7fff4b460da0) at ../ins-1.35.PR1.2-linux-x86_64/include/boost-1_35/boost/intrusive/rbtree_algorithms.hpp:1081 #5 0x00000000004053c5 in boost::intrusive::rbtree_algorithms<boost::intrusive::detail::rbtree_node_traits<boost::interprocess::offset_ptr<void> ::link_and_balance (z=@0x7fff4b460ed0, par=@0x7fff4b460ec0, left=true, header=@0x7fff4b460eb0) at ../ins-1.35.PR1.2-linux-x86_64/include/boost-1_35/boost/intrusive/rbtree_algorithms.hpp:968 #6 0x0000000000405dec in boost::intrusive::rbtree_algorithms<boost::intrusive::detail::rbtree_node_traits<boost::interprocess::offset_ptr<void>
::insert_equal_upper_bound<boost::intrusive::detail::key_node_ptr_compare<std::less<boost::interprocess::rbtree_best_fit<boost::interprocess::mutex_family, boost::interprocess::offset_ptr<void> >::block_ctrl>, boost::intrusive::set_base_hook<boost::intrusive::tag, (boost::intrusive::linking_policy)0, boost::interprocess::offset_ptr<void> ::value_traits<boost::interprocess::rbtree_best_fit<boost::interprocess::mutex_family, boost::interprocess::offset_ptr<void> >::block_ctrl> > > (h=@0x7fff4b460f70, new_node=@0x7fff4b460f60, comp={<boost::intrusive::detail::ebo_functor_holder<std::less<boost::interprocess::rbtree_best_fit<boost::interprocess::mutex_family, boost::interprocess::offset_ptr<void> >::block_ctrl> >> = {<boost::intrusive::detail::ebo_functor_holder_impl<std::less<boost::interprocess::rbtree_best_fit<boost::interprocess::mutex_family, boost::interprocess::offset_ptr<void> >::block_ctrl>,false>> = {<std::less<boost::interprocess::rbtree_best_fit<boost::interprocess::mutex_family, boost::interprocess::offset_ptr<void> >::block_ctrl>> = {<> = {<No data fields>}, <No data fields>}, <No data fields>}, <No data fields>}, <No data fields>}) at ../ins-1.35.PR1.2-linux-x86_64/include/boost-1_35/boost/intrusive/rbtree_algorithms.hpp:705 #7 0x0000000000405fd6 in boost::intrusive::rbtree<boost::intrusive::set_base_hook<boost::intrusive::tag, (boost::intrusive::linking_policy)0, boost::interprocess::offset_ptr<void> ::value_traits<boost::interprocess::rbtree_best_fit<boost::interprocess::mutex_family, boost::interprocess::offset_ptr<void> >::block_ctrl>, std::less<boost::interprocess::rbtree_best_fit<boost::interprocess::mutex_family, boost::interprocess::offset_ptr<void> >::block_ctrl>, true, unsigned long>::insert_equal_upper_bound (this=0x2aaaaaaad038, value=@0x2aaaaaaad120) at ../ins-1.35.PR1.2-linux-x86_64/include/boost-1_35/boost/intrusive/rbtree.hpp:367 #8 0x0000000000406048 in boost::intrusive::multiset<boost::intrusive::set_base_hook<boost::intrusive::tag, (boost::intrusive::linking_policy)0, boost::interprocess::offset_ptr<void> ::value_traits<boost::interprocess::rbtree_best_fit<boost::interprocess::mutex_family, boost::interprocess::offset_ptr<void> >::block_ctrl>, std::less<boost::interprocess::rbtree_best_fit<boost::interprocess::mutex_family, boost::interprocess::offset_ptr<void> >::block_ctrl>, true, unsigned long>::insert (this=0x2aaaaaaad038, value=@0x2aaaaaaad120) at ../ins-1.35.PR1.2-linux-x86_64/include/boost-1_35/boost/intrusive/set.hpp:1229 #9 0x000000000040653b in boost::interprocess::rbtree_best_fit<boost::interprocess::mutex_family, boost::interprocess::offset_ptr<void> >::priv_add_segment (this=0x2aaaaaaad010, addr=0x2aaaaaaad120, size=4712) at ../ins-1.35.PR1.2-linux-x86_64/include/boost-1_35/boost/interprocess/mem_algo/rbtree_best_fit.hpp:534 #10 0x0000000000406a3b in rbtree_best_fit (this=0x2aaaaaaad010, size=4984, extra_hdr_bytes=152) at ./ins-1.35.PR1.2-linux-x86_64/include/boost-1_35/boost/interprocess/mem_algo/rbtree_best_fit.hpp:418 #11 0x0000000000406af4 in segment_manager_base (this=0x2aaaaaaad010, size=4984, reserved_bytes=152) at ../ins-1.35.PR1.2-linux-x86_64/include/boost-1_35/boost/interprocess/segment_manager.hpp:95 #12 0x0000000000406ca5 in segment_manager (this=0x2aaaaaaad010, size=4984) at ../ins-1.35.PR1.2-linux-x86_64/include/boost-1_35/boost/interprocess/segment_manager.hpp:384 #13 0x0000000000406e2d in boost::interprocess::detail::basic_managed_memory_impl<char, boost::interprocess::rbtree_best_fit<boost::interprocess::mutex_family, boost::interprocess::offset_ptr<void> >, boost::interprocess::iset_index>::create_impl (this=0x7fff4b461450, addr=0x2aaaaaaad010, size=4984) at ../ins-1.35.PR1.2-linux-x86_64/include/boost-1_35/boost/interprocess/detail/managed_memory_impl.hpp:138 #14 0x0000000000407007 in boost::interprocess::detail::create_open_func<boost::interprocess::detail::basic_managed_memory_impl<char, boost::interprocess::rbtree_best_fit<boost::interprocess::mutex_family, boost::interprocess::offset_ptr<void> >, boost::interprocess::iset_index> >::operator() (this=0x7fff4b461350, addr=0x2aaaaaaad010, size=4984, created=true) at ../ins-1.35.PR1.2-linux-x86_64/include/boost-1_35/boost/interprocess/detail/managed_memory_impl.hpp:712 #15 0x00000000004076ed in boost::interprocess::detail::managed_open_or_create_impl<boost::interprocess::shared_memory_object,
(gdb) bt #0 0x000000000040202c in boost::interprocess::offset_ptr<boost::intrusive::detail::compact_rbtree_node<boost::interprocess::offset_ptr<void> true>::priv_open_or_create<boost::interprocess::detail::create_open_func<boost::interprocess::detail::basic_managed_memory_impl<char, boost::interprocess::rbtree_best_fit<boost::interprocess::mutex_family, boost::interprocess::offset_ptr<void> >, boost::interprocess::iset_index> > > (this=0x7fff4b461458, type=boost::interprocess::detail::DoCreateOrOpen, size=5000, mode=boost::interprocess::read_write, addr=0x0, construct_func={m_frontend = 0x7fff4b461450, m_type = boost::interprocess::detail::DoCreateOrOpen}) at ./ins-1.35.PR1.2-linux-x86_64/include/boost-1_35/boost/interprocess/detail/managed_open_or_create_impl.hpp:364 #16 0x0000000000407dc3 in managed_open_or_create_impl<boost::interprocess::detail::create_open_func<boost::interprocess::detail::basic_managed_memory_impl<char, boost::interprocess::rbtree_best_fit<boost::interprocess::mutex_family, boost::interprocess::offset_ptr<void> >, boost::interprocess::iset_index> > > (this=0x7fff4b461458, name=0x408707 "TEST_SEGMENT", size=5000, mode=boost::interprocess::read_write, addr=0x0, construct_func=@0x7fff4b4613f0) at ../ins-1.35.PR1.2-linux-x86_64/include/boost-1_35/boost/interprocess/detail/managed_open_or_create_impl.hpp:154 #17 0x0000000000407f1c in basic_managed_shared_memory (this=0x7fff4b461450, open_or_create={<No data fields>}, name=0x408707 "TEST_SEGMENT", size=5000, addr=0x0) at ../ins-1.35.PR1.2-linux-x86_64/include/boost-1_35/boost/interprocess/managed_shared_memory.hpp:89 #18 0x00000000004017d6 in main (argc=1, argv=0x7fff4b461568) at shm_test.cc:8 (gdb) Ion Gaztañaga wrote:
Glenn Schrader wrote:
If I compile the attached file with a version of trunk current as of a few days ago it works fine on 32 bit Linux but produces a seg fault on 64 bit Linux. An older version of 1.34+ from a few months ago works fine.
I configured using:
./configure --prefix=$HOME/boost_test --without-libraries=python,serialization,graph,test
Thanks for reporting it. I have no access to a 64 bit machine right noow but I'll try to get it ASAP. The example seems simple. I think you've already tested it, but are you sure it's a segfault and not an exception being thrown? If so, could you launch it with the debugger to see where the error is notified? Regards,
Ion _______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Glenn Schrader wrote:
Using gdb I get the following seg fault. I also included a backtrace.
Thanks for the info. Unfortunately, the lap from work (Centrino Core Duo) were I wanted to install 64 bit linux is not a 64bit processor (it seems that Core Duo 2 is 64 bit but no Core Duo), so I really don't know how to starting solving this :-( Regards, Ion
Ion, I will attempt to find the check in date where this problem first showed up. This ought to at least narrow down the possibilities. -glenn Ion Gaztañaga wrote:
Glenn Schrader wrote:
Using gdb I get the following seg fault. I also included a backtrace.
Thanks for the info. Unfortunately, the lap from work (Centrino Core Duo) were I wanted to install 64 bit linux is not a 64bit processor (it seems that Core Duo 2 is 64 bit but no Core Duo), so I really don't know how to starting solving this :-(
Regards,
Ion _______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
::get_pointer() and that was called a few layers underneath boost::interprocess::rbtree_best_fit<boost::interprocess::mutex_family, boost::interprocess::offset_ptr<void> >::priv_add_segment( ). Also note
Ion, I've narrowed this down to some checkins that were done on May 5th. The following files were added on that date: ln% diff -x .svn -I '^ *[/][/]' -Bpqr bld-2007-05-04/trunk/boost/interprocess bld-2007-05-05/trunk/boost/interprocess | grep Only Only in bld-2007-05-04/trunk/boost/interprocess: Attic Only in bld-2007-05-05/trunk/boost/interprocess/allocators: adaptive_pool.hpp Only in bld-2007-05-05/trunk/boost/interprocess/allocators: cached_adaptive_pool.hpp Only in bld-2007-05-05/trunk/boost/interprocess/allocators/detail: adaptive_node_pool.hpp Only in bld-2007-05-05/trunk/boost/interprocess/allocators/detail: node_tools.hpp Only in bld-2007-05-05/trunk/boost/interprocess/allocators: private_adaptive_pool.hpp Only in bld-2007-05-04/trunk/boost/interprocess/containers: Attic Only in bld-2007-05-04/trunk/boost/interprocess/containers/detail: Attic Only in bld-2007-05-05/trunk/boost/interprocess/containers/detail: tree.hpp Only in bld-2007-05-05/trunk/boost/interprocess/detail/Attic: gcd_lcm.hpp Only in bld-2007-05-04/trunk/boost/interprocess/detail/Attic: multi_segment_services.hpp Only in bld-2007-05-05/trunk/boost/interprocess/detail: gcd_lcm.hpp Only in bld-2007-05-05/trunk/boost/interprocess/detail: ptime_wrk.hpp Only in bld-2007-05-05/trunk/boost/interprocess/indexes: iset_index.hpp Only in bld-2007-05-05/trunk/boost/interprocess/indexes: iunordered_set_index.hpp Only in bld-2007-05-05/trunk/boost/interprocess: managed_windows_shared_memory.hpp Only in bld-2007-05-04/trunk/boost/interprocess/mem_algo: Attic Only in bld-2007-05-05/trunk/boost/interprocess/mem_algo: rbtree_best_fit.hpp Only in bld-2007-05-05/trunk/boost/interprocess/smart_ptr/detail: sp_counted_base_atomic.hpp Only in bld-2007-05-05/trunk/boost/interprocess/smart_ptr: segment_deleter.hpp Only in bld-2007-05-05/trunk/boost/interprocess: windows_shared_memory.hpp The function where the seg fault is happening is boost::interprocess::offset_ptr<boost::intrusive::detail::compact_rbtree_node<boost::interprocess::offset_ptr<void> that rbtree_best_fit.hpp is in the above list. Even so, the seg fault is happening inside of a call to a multiset insert so perhaps its actually the multiset class that has the problem. Does this ring any bells? In any event, I'll keep looking... Glenn Schrader wrote:
Ion,
I will attempt to find the check in date where this problem first showed up. This ought to at least narrow down the possibilities.
-glenn
Ion Gaztañaga wrote:
Glenn Schrader wrote:
Using gdb I get the following seg fault. I also included a backtrace.
Thanks for the info. Unfortunately, the lap from work (Centrino Core Duo) were I wanted to install 64 bit linux is not a 64bit processor (it seems that Core Duo 2 is 64 bit but no Core Duo), so I really don't know how to starting solving this :-(
Regards,
Ion _______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
I found it. The problem is in offset_ptr.hpp. A diff against the current trunk is below. Note that I just moved the ~. The problem is that gcc is treating pointers as 64 bit values but int and unsigned as 32 bit values. Just to verify this, when I run the program ... #include <iostream> using namespace std; unsigned u; int i; int main() { cout << sizeof(u) << ' ' << sizeof(&u) << endl; cout << sizeof(i) << ' ' << sizeof(&i) << endl; cout << sizeof(size_t) << endl; } ... the output is ... 4 8 4 8 8 In the original code the ~ only complemented the 32 bit unsigned value. When the value was expanded into a size_t the value isn't sign extended so the upper 32 bits wind up being zero. Since this is used as an address mask the upper half of the address is zeroed. This is gcc 4.1.1 on kernel 2.6.20-1.2948.fc6 in case anybody is interested. -glenn % svn diff Index: boost/interprocess/offset_ptr.hpp =================================================================== --- boost/interprocess/offset_ptr.hpp (revision 39447) +++ boost/interprocess/offset_ptr.hpp (working copy) @@ -384,7 +384,7 @@ typedef boost::interprocess::offset_ptr<T> pointer; static pointer get_pointer(const pointer &n) - { return (T*)(std::size_t(n.get()) & std::size_t(~2u)); } + { return (T*)(std::size_t(n.get()) & ~std::size_t(2u)); } static void set_pointer(pointer &n, pointer p) { n = (T*)(std::size_t(p.get()) | (std::size_t(n.get()) & std::size_t(2u))); } % Glenn Schrader wrote:
Ion,
I will attempt to find the check in date where this problem first showed up. This ought to at least narrow down the possibilities.
-glenn
Ion Gaztañaga wrote:
Glenn Schrader wrote:
Using gdb I get the following seg fault. I also included a backtrace.
Thanks for the info. Unfortunately, the lap from work (Centrino Core Duo) were I wanted to install 64 bit linux is not a 64bit processor (it seems that Core Duo 2 is 64 bit but no Core Duo), so I really don't know how to starting solving this :-(
Regards,
Ion _______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Glenn Schrader wrote:
I found it. The problem is in offset_ptr.hpp. A diff against the current trunk is below. Note that I just moved the ~. The problem is that gcc is treating pointers as 64 bit values but int and unsigned as 32 bit values.
Thank you *very* much for the patch. It seems that finding the bug hasn't been easy (the bug was so deep in the code), so sorry if this has been too hard to find. I've already committed your patch. Regards, Ion
Ion Gaztañaga wrote:
Thank you *very* much for the patch. It seems that finding the bug hasn't been easy (the bug was so deep in the code), so sorry if this has been too hard to find. I've already committed your patch.
I've found a similar bug in boost/intrusive/pointer_plus_bit.hpp. I don't think that will affect most Interprocess users, but I recommend updating your Boost.Intrusive snapshot. Regards, Ion
Ion, I've run across an issue in the shared memory initialization sequence. In managed_open_or_create_impl.hpp at around line 291 there is a a try / catch that detects if creating a shared memory segment fails due to it already existing. This fails to work for programs compiled without exceptions. What happens is that any thrown exception causes an immediate program exit so the try / catch handler never gets a chance to act. Glenn Schrader - MITLL
Glenn Schrader wrote:
Ion,
I've run across an issue in the shared memory initialization sequence. In managed_open_or_create_impl.hpp at around line 291 there is a a try / catch that detects if creating a shared memory segment fails due to it already existing. This fails to work for programs compiled without exceptions. What happens is that any thrown exception causes an immediate program exit so the try / catch handler never gets a chance to act.
Glenn Schrader - MITLL _______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
For the moment, Interprocess does not support programs compiled without exceptions. The issue is hard to solve, because we need to handle memory exhaustion issues on all shared memory containers, and there is no standard interface for that. I have plans to investigate how could I change Interprocess so that it can be used with no exceptions. I would need to start offering non-throwing alternatives in all constructors and maybe some hooks on STL-like allocators to handle memory exhaustion issues. But I'm afraid this effort won't be soon, because I'm buried on several issues. If you have any suggestion to make Interprocess compatible with no exceptions, let me know. Regards, Ion
Well, I hacked up a version that used status flags to return the open failure rather than an exception and it works find. Of course if there ever was any out-of-memory condition (etc) then it would just terminate but I view that as just part of the contract you get when you turn exceptions off then use an API that might throw. In practice this level of functionality is usable. On a more philosophical note, I don't think that exceptions are the "right" way to implement this. Exceptions are best used to signal exceptional conditions and not as a way to control normal program flow. An open failure, rather than being exceptional, is the most commonly expected case since only one process will actually not throw. My $0.02. -glenn Ion Gaztañaga wrote:
Glenn Schrader wrote:
Ion,
I've run across an issue in the shared memory initialization sequence. In managed_open_or_create_impl.hpp at around line 291 there is a a try / catch that detects if creating a shared memory segment fails due to it already existing. This fails to work for programs compiled without exceptions. What happens is that any thrown exception causes an immediate program exit so the try / catch handler never gets a chance to act.
Glenn Schrader - MITLL _______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
For the moment, Interprocess does not support programs compiled without exceptions. The issue is hard to solve, because we need to handle memory exhaustion issues on all shared memory containers, and there is no standard interface for that.
I have plans to investigate how could I change Interprocess so that it can be used with no exceptions. I would need to start offering non-throwing alternatives in all constructors and maybe some hooks on STL-like allocators to handle memory exhaustion issues. But I'm afraid this effort won't be soon, because I'm buried on several issues.
If you have any suggestion to make Interprocess compatible with no exceptions, let me know.
Regards,
Ion
_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Glenn Schrader wrote:
Well, I hacked up a version that used status flags to return the open failure rather than an exception and it works find. Of course if there ever was any out-of-memory condition (etc) then it would just terminate but I view that as just part of the contract you get when you turn exceptions off then use an API that might throw. In practice this level of functionality is usable. On a more philosophical note, I don't think that exceptions are the "right" way to implement this. Exceptions are best used to signal exceptional conditions and not as a way to control normal program flow. An open failure, rather than being exceptional, is the most commonly expected case since only one process will actually not throw. My $0.02.
That was the case for the original Shmem (Interprocess parent) library but in the review Boosters decided that exceptions were the way to go. The problem with out-of memory is that getting out of memory in the shared memory segment is more common than out-of-memory in the heap. And crashing the program when the shared memory is full is not a good idea, IMHO. What I could do is to add non-throwing overloads, but I don't know when I'll have time to do it. I would want to make the whole library compatible with disabled exceptions (including containers and other classes). I know someday I'll do a non-throwing version of Interprocess because I not a big fan of exceptions, but I don't know when ;-) Regards, Ion
participants (2)
-
Glenn Schrader -
Ion Gaztañaga