[interprocess] [bimap] Using raw pointers
Hello, Currently Boost.Bimap does not work with interprocess allocators because it uses references to connect the views and the core. I want to change this but avoid any performance hit and I am wondering what is the best way to do it. It will be easy if Bimap interface will use .left() and .right() functions to access the map views (then I could just init the views at each call like Boost.MultiIndex get() index function does) but I am stuck with the left and right members (and I actually like them). So, what I want to be able to do is the following: template<class Bimap> class left_type { Bimap* core; public: left_type(Bimap* c) : core(c) {}; ... }; template<class Bimap> class right_type {...}; template<class LeftKey,class RightKey> class bimap { public: left_type<bimap> left; right_type<bimap> right; bimap() : left(this), right(this) {} }; This will not work because I am not using the proper offset_pointer abstraction, but I would love to be able to avoid it. In the case of Bimap, these pointers are very easy to maintain. Will it be possible to add an extension point to Boost.Interprocess to allow this idiom? Something like adding a call to the end of priv_find_impl of segment_manager so user classes can rewire themselves when they are loaded from shared memory. For example: template<class T> void rewire_after_loading_from_shared_memory(T* t) { } ... template <class T> std::pair<T*, size_type> priv_find_impl (const CharType* name, bool lock) { // find ret pointer T* t_ret = static_cast<T*>(ret) ; if( t_ret ) rewire_after_loading_from_shared_memory( t_ret ); return std::pair<T*, size_type>(t_ret ,sz); } ... Or maybe a better place. Then I will be able to rewire the map views pointers to the core if they are different from it: template<class L, class R> void rewire_after_loading_from_shared_memory(bimap<L,R>* t) { t->rewire(); } If this is not possible because inherent limitations of the interprocess mechanisms, do you know a good way to make Boost.Bimap compatible with its allocators? Thanks, Best regards Matias
El 25/01/2013 20:05, Matias Capeletto escribió:
Hello,
Currently Boost.Bimap does not work with interprocess allocators because it uses references to connect the views and the core. I want to change this but avoid any performance hit and I am wondering what is the best way to do it. It will be easy if Bimap interface will use .left() and .right() functions to access the map views (then I could just init the views at each call like Boost.MultiIndex get() index function does) but I am stuck with the left and right members (and I actually like them). So, what I want to be able to do is the following:
template<class Bimap> class left_type { Bimap* core; public: left_type(Bimap* c) : core(c) {}; ... };
If Bimap exports a pointer type (say Bimap::pointer or similar) that can be offset_ptr or raw pointer (I think multiindex exports it), then you just need to use: typedef boost::pointer_to_other <typename Bimap::pointer, Bimap>::type bimap_ptr; typedef boost::intrusive::pointer_traits <typename Bimap::pointer>:: rebind_pointer<Bimap>::type bimap_ptr; and use it template<class Bimap> class left_type { typedef boost::pointer_to_other <typename Bimap::pointer, Bimap>::type bimap_ptr; bimap_ptr core; public: left_type(const bimap_ptr &c) : core(c) {}; }; Best, Ion
On Fri, Jan 25, 2013 at 9:30 PM, Ion Gaztañaga <igaztanaga@gmail.com> wrote:
If Bimap exports a pointer type (say Bimap::pointer or similar) that can be offset_ptr or raw pointer (I think multiindex exports it), then you just need to use:
typedef boost::pointer_to_other <typename Bimap::pointer, Bimap>::type bimap_ptr;
typedef boost::intrusive::pointer_traits <typename Bimap::pointer>:: rebind_pointer<Bimap>::type bimap_ptr;
Thanks for the answer Ion. Ok, so doing this I will only pay for the offset_ptr in case the allocator needs it. That is good. But still when users will select an interprocess allocator they will be paying for every call to the views: left_type& left = bm.left; left.size(); // needs to use offset_ptr What MultiIndex is doing is creating the index views with a pointer to (this) every time the user ask them. I could do the same if I deprecate .left and .right and include get_left() and get_right() functions. But as I said I will like to keep the current interface... is something like the rewire extension impossible? (I am afraid it is if you are pointing to read only memory). Or maybe some another trick to point the members to the core. Best Matias
El 25/01/2013 21:47, Matias Capeletto escribió:
On Fri, Jan 25, 2013 at 9:30 PM, Ion Gaztañaga <igaztanaga@gmail.com> wrote:
If Bimap exports a pointer type (say Bimap::pointer or similar) that can be offset_ptr or raw pointer (I think multiindex exports it), then you just need to use:
typedef boost::pointer_to_other <typename Bimap::pointer, Bimap>::type bimap_ptr;
typedef boost::intrusive::pointer_traits <typename Bimap::pointer>:: rebind_pointer<Bimap>::type bimap_ptr;
Thanks for the answer Ion. Ok, so doing this I will only pay for the offset_ptr in case the allocator needs it. That is good.
But still when users will select an interprocess allocator they will be paying for every call to the views:
left_type& left = bm.left; left.size(); // needs to use offset_ptr
Have this problem also with iterators. Containers with interprocess allocators will return an iterator that can be placed in shared memory (contains offset_ptr) but when you use it to iterate in your process, you pay the price of offset_ptr. I think multiindex also pays this price.
What MultiIndex is doing is creating the index views with a pointer to (this) every time the user ask them. I could do the same if I deprecate .left and .right and include get_left() and get_right() functions. But as I said I will like to keep the current interface... is something like the rewire extension impossible? (I am afraid it is if you are pointing to read only memory).
I don't fully understand what do you achieve with "rewire", can you elaborate, please? Best, Ion
On Fri, Jan 25, 2013 at 11:49 PM, Ion Gaztañaga <igaztanaga@gmail.com> wrote:
El 25/01/2013 21:47, Matias Capeletto escribió:
But still when users will select an interprocess allocator they will be paying for every call to the views:
left_type& left = bm.left; left.size(); // needs to use offset_ptr
Have this problem also with iterators. Containers with interprocess allocators will return an iterator that can be placed in shared memory (contains offset_ptr) but when you use it to iterate in your process, you pay the price of offset_ptr. I think multiindex also pays this price.
Ok, that is interesting. Maybe I should just use offset_ptr then, if every time I iterate over the container I am using them anyways.
What MultiIndex is doing is creating the index views with a pointer to (this) every time the user ask them. I could do the same if I deprecate .left and .right and include get_left() and get_right() functions. But as I said I will like to keep the current interface... is something like the rewire extension impossible? (I am afraid it is if you are pointing to read only memory).
I don't fully understand what do you achieve with "rewire", can you elaborate, please?
Maybe you do not understand this because is a crazy idea in the context of Interprocess. I looked at your code and in the segment_manager, before you return the user the found Bimap* I wanted a user extension point to be called on the pointer: if( t_ret ) rewire_after_loading_from_shared_memory( t_ret ); return std::pair<T*, size_type>(t_ret ,sz); And I will do something like (this function being a friend of the views): template<class L, class R> void rewire_after_loading_from_shared_memory(bimap<L,R>* t) { t->left.rewire(t); t->right.rewire(t); } In left_type and right_type private section: void rewire(Core* c) { if(core!=c) core=c; } Event if this could work, the question will be if we want to allow users to place views in shared memory. For Bimap, I think it doesn't make sense... you should always stored the bimap and then get the views. But now I looked better at MultiIndex implementation and it is actually pushing the index as part of the main container hierarchy using CRTP to access the core, something I thought about yesterday also. I think I will end up doing the same for Bimap then. Thanks, Best Matias
El 26/01/2013 7:32, Matias Capeletto escribió:
But now I looked better at MultiIndex implementation and it is actually pushing the index as part of the main container hierarchy using CRTP to access the core, something I thought about yesterday also. I think I will end up doing the same for Bimap then.
That's quite clever ;-) I think it's a good solution for your problem. Best, Ion
On Sat, Jan 26, 2013 at 8:50 AM, Ion Gaztañaga <igaztanaga@gmail.com> wrote:
El 26/01/2013 7:32, Matias Capeletto escribió:
But now I looked better at MultiIndex implementation and it is actually pushing the index as part of the main container hierarchy using CRTP to access the core, something I thought about yesterday also. I think I will end up doing the same for Bimap then.
That's quite clever ;-) I think it's a good solution for your problem.
Well, unfortunately, I am back to square one. Again, I will not have any problem if the views will be accessed from a left() and right() functions but I can not find a way to get from the view data member to the core that is not undefined in C++. For reference, I have to get from the left data member to the bimap parent: class bimap { ... left_map_view left; ... } ; Any attempt to use the offset of the member to get to the bimap class seems to be forbidden by the standard. So, I am back to using offset_ptr for the map_view data members and maybe at one point think about migrating to function-accessed views to avoid it :( Best regards Matias
El 27/01/2013 9:02, Matias Capeletto escribió:
Any attempt to use the offset of the member to get to the bimap class seems to be forbidden by the standard.
Yes, i'ts undefined behaviour in theory but not in practice. Boost.Intrusive has a useful function implemented for many compilers: http://www.boost.org/doc/libs/1_52_0/boost/intrusive/parent_from_member.hpp It's similar to a "downcast" but using members. Like "static_cast", it does not work with virtual inheritance. In your case the pointer to member value can be a compile-time constant (so &Bimap::left can be passed as a template parameter). This constant can be part of the type "left_map" or the compile-time pointer to member could be obtained by left_map by a metafunction, call parent_from_member passing "this" and the compile-time pointer to member value, obtaining the pointer to Bimap. This way you can avoid storing any pointer to Bimap into left_map. Best, Ion
On Sun, Jan 27, 2013 at 9:39 AM, Ion Gaztañaga <igaztanaga@gmail.com> wrote:
El 27/01/2013 9:02, Matias Capeletto escribió:
Any attempt to use the offset of the member to get to the bimap class seems to be forbidden by the standard.
Yes, i'ts undefined behaviour in theory but not in practice. Boost.Intrusive has a useful function implemented for many compilers:
http://www.boost.org/doc/libs/1_52_0/boost/intrusive/parent_from_member.hpp
This. It is exactly what I was looking for yesterday... any chance to make this more visible? It is a very handy tool.
In your case the pointer to member value can be a compile-time constant (so &Bimap::left can be passed as a template parameter). This constant can be part of the type "left_map" or the compile-time pointer to member could be obtained by left_map by a metafunction, call parent_from_member passing "this" and the compile-time pointer to member value, obtaining the pointer to Bimap. This way you can avoid storing any pointer to Bimap into left_map.
Awesome. This is what I wanted to do, but I failed because the basic piece in the puzzle was missing. Thanks a lot, I didn't actually expected you to directly give it to me :) I will do the changes then so Bimap can be used with Interprocess allocators. Best regards Matias
On Sun, Jan 27, 2013 at 10:36 PM, Ion Gaztañaga <igaztanaga@gmail.com> wrote:
El 27/01/2013 19:00, Matias Capeletto escribió:
This. It is exactly what I was looking for yesterday... any chance to make this more visible? It is a very handy tool.
What do you mean with "more visible", maybe try to put it into boost/utility?
It can be. It would have helped me because I looked there to see if there was something to help me. Google didn't helped either... I looked for "parent from member C++" and "boost parent from member C++" yesterday and there was nothing to point me to your work.
2013/1/26 Matias Capeletto <matias.capeletto@gmail.com>
On Fri, Jan 25, 2013 at 9:30 PM, Ion Gaztañaga <igaztanaga@gmail.com> wrote:
If Bimap exports a pointer type (say Bimap::pointer or similar) that can be offset_ptr or raw pointer (I think multiindex exports it), then you just need to use:
typedef boost::pointer_to_other <typename Bimap::pointer, Bimap>::type bimap_ptr;
typedef boost::intrusive::pointer_traits <typename Bimap::pointer>:: rebind_pointer<Bimap>::type bimap_ptr;
Thanks for the answer Ion. Ok, so doing this I will only pay for the offset_ptr in case the allocator needs it. That is good.
But still when users will select an interprocess allocator they will be paying for every call to the views:
left_type& left = bm.left; left.size(); // needs to use offset_ptr
What MultiIndex is doing is creating the index views with a pointer to (this) every time the user ask them. I could do the same if I deprecate .left and .right and include get_left() and get_right() functions. But as I said I will like to keep the current interface... is something like the rewire extension impossible? (I am afraid it is if you are pointing to read only memory).
Or maybe some another trick to point the members to the core.
Is it possible for you to make left & right in a union and put it in the front of bimap (as 1st base or member) so you can cast (this) to bimap? Not sure about strict-aliasing though...
On Sat, Jan 26, 2013 at 4:46 AM, TONGARI <tongari95@gmail.com> wrote:
2013/1/26 Matias Capeletto <matias.capeletto@gmail.com>
Or maybe some another trick to point the members to the core.
Is it possible for you to make left & right in a union and put it in the front of bimap (as 1st base or member) so you can cast (this) to bimap?
Not sure about strict-aliasing though...
Look at my mail to Ion, it seems that MultiIndex was putting the indexes as part of the hierarchy to achieve this.
participants (3)
-
Ion Gaztañaga -
Matias Capeletto -
TONGARI