December 2006 - Boost-users - lists.preview.boost.org

Re: [Boost-users] Problem with multi_index and shared_ptr
by johneddy101＠comcast.net 20 Dec '06

20 Dec '06

Joaquin, Thanks for the quick response and the quick fix. I inserted your new code and everything seems to be fine. Prior to inserting the code, I was doing 150,000 insertions and subsequent erasures from a container. It would always fault at some point during the erasures but exactly where was unreliable. It does not seem to fault now even if I bump it up to 1,500,000 which is very strong evidence that your solution has worked. Thanks again for the great lib, John Joaquín Mª López Muñoz ha escrito: Hello John, johneddy101 <at> comcast.net ha escrito: I'm running into a problem while using hashed indices in a container storing shared_ptr's. The problem is occuring in the hashed_index::erase(key) method in both the 1_33_1 release and the CVS head version. This is what the problem seems to be: It is possible that the call to erase (or final_erase_) occurring within the do loop causes destruction of the passed in key object. The key object (k) is then used in a comparison in the while portion of the loop (y!=x&&eq(k, key())). The default predicate uses == in this comparison but one of the items (the left hand side) will have already been destroyed. Umm... Thank you for spotting this, indeed the key passed to erase() can become invalid during the process if it physically belongs to one of the elements to be erased. Matter of fact, the problem is not specifically related to shared_ptr's. I don't know if it's too late to try to fix this on time for Boost 1.34; in case I find the time and permission to do it I'll let you know. An update on this. The following rewrite of erase(key) in hashed_index.hpp should clear the problem: size_type erase(key_param_type k) { BOOST_MULTI_INDEX_HASHED_INDEX_CHECK_INVARIANT; size_type s=0; std::size_t buc=buckets.position(hash(k)); hashed_index_node_impl* x=buckets.at(buc); hashed_index_node_impl* y=x->next(); while(y!=x){ if(eq(k,key(node_type::from_impl(y)->value()))){ bool b; do{ hashed_index_node_impl* z=y->next(); b=z!=x&&eq( key(node_type::from_impl(y)->value()), key(node_type::from_impl(z)->value())); this->final_erase_( static_cast<final_node_type*>(node_type::from_impl(y))); y=z; ++s; }while(b); break; } y=y->next(); } return s; } Would you mind pasting this into your local copy of hashed_index.hpp (CVS version) and report whether your prior problems get solved? I'm writing a testcase for the issue and running my entire regression tests, if everything goes OK I think I'll be able to commit this into the CVS before the week ends. Thank you, Joaquín M López Muñoz Telefónica, Investigación y Desarrollo

1 0

lambda parse tree access
by Sascha Krissler 20 Dec '06

20 Dec '06

Hello, i am writing a multiple dispatch framework and as i am feeling scientific i am gonna try to add a predicate dispatch fearure to it. For that i need to build big nested expression template instances that represent the predicates, so that i cant evaluate them at the time of method invocation. As an additional feature i want to sort the predicates for specifity (one predicate is true at a subset of cases of another predicate and is as such more specific) and for that i need access to the parse tree of a lambda expression. Is there documentation available on this topic and does lambda have a public interface for that? Does it make sense to use lambda as a library for writing expression template algorithms? Thanks in advance

5 6

Re: [Boost-users] Problem with multi_index and shared_ptr
by Joaquín Mª López Muñoz 20 Dec '06

20 Dec '06

Hello John, johneddy101(a)comcast.net ha escrito: > I'm running into a problem while using hashed indices in a container > storing shared_ptr's. > The problem is occuring in the hashed_index::erase(key) method in both > the 1_33_1 > release and the CVS head version. This is what the problem seems to > be: > > It is possible that the call to erase (or final_erase_) occurring > within the do loop > causes destruction of the passed in key object. The key object (k) is > then used in a > comparison in the while portion of the loop (y!=x&&eq(k, key())). The > default predicate uses > == in this comparison but one of the items (the left hand side) will > have already been destroyed. Umm... Thank you for spotting this, indeed the key passed to erase() can become invalid during the process if it physically belongs to one of the elements to be erased. Matter of fact, the problem is not specifically related to shared_ptr's. I don't know if it's too late to try to fix this on time for Boost 1.34; in case I find the time and permission to do it I'll let you know. > I'm afraid that I don't know a good solution to this problem. I see > from your commit > logs on this file that there is a problem using something like is done > in the ordered_indices > (deleting equal_range items) whereby it increases the complexity. I > can see from the > implementation of equal_range on hashed indices that this is a > potentially expensive > operation for non-unique collections with many collisions. However, > since speed is a > secondary consideration for me, this is the solution I'm using for > now. > Do you mean that instead of c.erase(k); you do the following? std::pair<iterator,iterator> p=c.equal_range(k); c.erase(p.first,p.second); This is a viable solution. The potential performance problem you point out is actually present in both methods. On the other hand, the different problem I talk about in the commit logs only affects to erase(first,last), not erase(k), and is exposed here: http://open-std.org/jtc1/sc22/wg21/docs/papers/2006/n2023.pdf An alternative solution is to do the following: c.erase(key_type(k)); i.e. doing an external copy of the key before calling erase(k). This is the solution I'd use if your keys are cheap to copy. > Thanks, > > John Thank you for reporting, good luck with your project. Joaquín M López Muñoz Telefónica, Investigación y Desarrollo

1 1

[boost.multiindex] Recommended way of having a list of keys to a multi-index container
by Charles Tolman 20 Dec '06

20 Dec '06

Hi, Boost newbie question regarding indexing of multi_index_containers. First a great big thank you to Joaquín M López Muñoz for sorting this library out. Lovely job. OK I have a question about indexing. Probably one for Joaquin. I have a structure that can be indexed on the following keys: 0: identity -- unique - this is fine. 1: int position; -- non_unique - this is fine. 2: int priority; -- non_unique - this is also fine. 3: vector<MyStringType> -- non_unique - but NOT fine. So number 3 is the problem. Am I right in thinking that the multi_index_container does not handle this case? i.e. where you would want a given SINGLE element to be indexed on a variable (unknown) number of keys, in this case as many strings as are in the vector. Although strictly they are not actually separate keys, there is just one non-unique key, but a given element can have multiple instances for this one key. Of course I can workaround this by having another structure to handle this or alternatively wrap up the element a multiple number of times, as many as required by the length of the vector of strings. But I was wondering if there are any ways of handling this in the basic library. Apologies if I have missed something and it is in an FAQ somewhere, although I have done an initial search. TIA Charles Tolman

2 2

[Serialization] Boost 1.33 serialization const problem
by vermaaso 20 Dec '06

20 Dec '06

Hello list, currently I am trying to get serialization to work. It worked well when I was using boost 1.32, but since I upgraded to 1.33 I'm having some problems. In short; I have an abstract base class and a derived class. I try to serialize/deserialize the derived class through a pointer to the base class. This worked well when I was using 1.32. When I upgraded to 1.33 I got the static assert which trapped the 'saving non const error' (I hope this makes sense ..). So I fixed it by following instructions from other posts dealing with the same problem (basically just by changing 'base *b' to 'const base * const b'). The fix resolved the compile error, only now I'm getting a segfault at runtime. The segfault happens during deserialization (when trying to 'reconstruct' the original object). The funny thing is, I've tested it on a system (suse enterprise system 9, gcc 3.3.3) and it chrashes. On another system (gcc version 4.0.3 (Ubuntu 4.0.3-1ubuntu5)) it works fine. I reproduced the problem in a very small program located at the end of this post. Maybe someone can have a look at the code and tell me whether there are obvious problems with my code or not. If I know for sure that my code should run correctly, I'll just move on to an other linux distro .. Thanks in advance, Oscar #include<boost/archive/text_oarchive.hpp> #include<boost/archive/text_iarchive.hpp> #include<boost/serialization/base_object.hpp> #include<boost/serialization/export.hpp> #include<iostream> #include<sstream> using namespace std; class base { public: virtual void do_it() const = 0; private: friend class boost::serialization::access; template<class Archive> void serialize(Archive & ar, const unsigned int){ cout << "serializing base" << endl; } }; class derived: public base { public: derived(int field1, string field2){ _field1 = field1; _field2 = field2; } virtual void do_it() const { cout << "doing it with " << _field1 << " and " << _field2 << endl; } private: friend class boost::serialization::access; derived(){}; virtual ~derived(){}; template<class Archive> void serialize(Archive & ar, const unsigned int){ cout << "derived being serialized" << endl; ar & boost::serialization::base_object<base>(*this); ar & _field1; ar & _field2; } int _field1; string _field2; }; BOOST_IS_ABSTRACT(base); BOOST_CLASS_EXPORT(derived); string serialize(const base * const b){ stringstream str; boost::archive::text_oarchive oa(str); oa << b; return str.str(); } base *deserialize(string s){ stringstream str(s); base *b; boost::archive::text_iarchive ia(str); ia >> b; return b; } int main(){ base *d1 = new derived(52,"a string"); d1->do_it(); string serialized = serialize(d1); base *d2 = deserialize(serialized); d2->do_it(); return 0; }

2 10

signals and threads
by Noah Roberts 20 Dec '06

20 Dec '06

The boost::signals library faq says that the library is not thread safe and is in fact dangerous. Does anyone know why? What are the implications of using the signals library in a threaded environment? Would I be better off implementing an event or observer architecture with boost::function instead? thanks

1 0

ptr_container: ptr_map.insert() not compiling?
by Duane Murphy 19 Dec '06

19 Dec '06

I'm trying to use boost::ptr_map<> and cannot get calls insert to compile. I'm on Mac OS X using gcc 3.3 with Xcode. Even the most basic code fails to compile: boost::ptr_map< in, int* > my_map; int * val = new int( 22 ); my_map.insert( 1, val ); This fails with: error: no matching function for call to `boost::ptr_map<int, int*, std::less<int>, boost::heap_clone_allocator, std::allocator<std::pair<const int, void*> > >::insert(int, int*&)' error: candidates are: std::pair<typename boost::ptr_container_detail::ptr_map_adapter_base<T, VoidPtrMap, CloneAllocator>::iterator, bool> boost::ptr_map_adapter<T, VoidPtrMap, CloneAllocator>::insert(typename boos t::ptr_container_detail::ptr_map_adapter_base<T, VoidPtrMap, CloneAllocator>::key_type&, typename boost::ptr_container_detail::ptr_map_adapter_base<T, VoidPtrMap, CloneAllocator>::value_type) [with T = int*, VoidPtrMap = std::map<int, void*, std::less<int>, std::allocator<std::pair<const int, void*> > >, CloneAllocator = boost::heap_clone_allocator] This looks like it's recommending the function that should be working. I've been poking at this for quite a while and cannot find a clue why this is not working. Thank you for the help, ...Duane ...Duane

4 6

[variant] make_recursive_variant and std::map
by Richard Crossley 19 Dec '06

19 Dec '06

I would like to be able to use a recursive variant similar to the way it's shown in the documentation but using a std::map instead of std::vector. i.e. boost::make_recursive_variant< int,std::map<float,boost::recursive_variant_> >::type var; Is there a way to do this? VC8.1 complains with: utility(54) : error C2079: 'std::pair<_Ty1,_Ty2>::second' uses undefined struct 'boost::recursive_variant_' with [ _Ty1=const float, _Ty2=boost::recursive_variant_ ] Regards, Richard.

2 2

Problem with multi_index and shared_ptr
by johneddy101＠comcast.net 19 Dec '06

19 Dec '06

I'm running into a problem while using hashed indices in a container storing shared_ptr's. The problem is occuring in the hashed_index::erase(key) method in both the 1_33_1 release and the CVS head version. This is what the problem seems to be: It is possible that the call to erase (or final_erase_) occurring within the do loop causes destruction of the passed in key object. The key object (k) is then used in a comparison in the while portion of the loop (y!=x&&eq(k, key())). The default predicate uses == in this comparison but one of the items (the left hand side) will have already been destroyed. I'm afraid that I don't know a good solution to this problem. I see from your commit logs on this file that there is a problem using something like is done in the ordered_indices (deleting equal_range items) whereby it increases the complexity. I can see from the implementation of equal_range on hashed indices that this is a potentially expensive operation for non-unique collections with many collisions. However, since speed is a secondary consideration for me, this is the solution I'm using for now. Thanks, John

1 0

Re: [Boost-users] Boost Serialization make_binary & XML/ASCII
by Javier Estrada 19 Dec '06

19 Dec '06

Terence, We use Spirit in a similar way to SAX: we read binary data from a serial port, assemble it into discrete units (messages) and push them through parser generated by Spirit, which in turn calls a function upon finding a match. Some of our messages are multi-part, so we need to keep some form of state that allows us to assemble the final message. I would imagine that there would be a way to break from the parsing. -----Original Message----- From: boost-users-bounces(a)lists.boost.org [mailto:boost-users-bounces@lists.boost.org] On Behalf Of Terence Wilson Sent: Saturday, December 09, 2006 2:55 PM To: boost-users(a)lists.boost.org Subject: Re: [Boost-users] Boost Serialization make_binary & XML/ASCII Robert, XML is normally parsed using a DOM or SAX parser. DOM reads the whole file into memory, SAX behaves like a recursive descent parser with callbacks to the client application. By placing the data block at the start of the file I should be able to get good performance from SAX or Spirit. Both would be good choices, however, I want to write some 'reference' code using standard tools since my work will be part of an SDK. Regards, Terence > -----Original Message----- > From: boost-users-bounces(a)lists.boost.org [mailto:boost-users- > bounces(a)lists.boost.org] On Behalf Of Robert Ramey > Sent: Saturday, December 09, 2006 2:10 PM > To: boost-users(a)lists.boost.org > Subject: Re: [Boost-users] Boost Serialization make_binary & XML/ASCII > > Well, now I'm out of my depth. Some have commented that the spirit > parser is slower than other xml parsers.I don't know. I would have > hoped that since spirit does a lot of the heavy lifting at compile > time, it would be pretty fast. I haven't seen too much data on this so I really don't know. > Any parser has to scan every character in the file so its not clear to > me that a SAX parser or any other can be know a priore to be faster > than any other one. > > My reason for using spirit was > > a) it was already part of boost > b) it was - after some learning curve - a good fit with what I wanted > to do. > c) well documented. > d) customizable - serialization only uses a portion of the full xml so > it seemed the most efficient. > e) all done at compile time so it wouldn't include dead code. > f) portability to all compilers boost supports. > g) By exercising a little care in code organization I was able to > arrange things so that the module containing the parsing didn't depend > on the rest of the program. So the long compile time is not an issue. > It is in the library and is only recompiled when the grammar changes. > > It is the last feature that suggests that you can easily use this to > do your own actions upon parsing the serialization library. > > After some initial pain figuring out how to use it, I have to say I > have been extremely pleased with this application of spirit. I never > wanted to do xml serialization as I felt it was a pain in the neck and > of relatively little utility in my view. I had anticipated a > maintainence nightmare so more and more obscure corners of xml syntax > were touched. I'm pleased to say this thing has been fantastic as far > as I'm concerned. After the intial one time pain - I haven't had to > touch it since 2002 - and this (through spirt 1.6x - still available) > is still compatible with Borland 5.51. And all the hacks required to > make this so portable are only compiled into the platforms that need > them. > > This has been one of the most significant implementations in making > the serialization library possible. (the other one would probably be mpl). > > So if this were my problem I would: > > a) Include the xml grammar and parser from the serialization library - > add my own actions. > b) finish my code. Really this I would expect it would be 100 lines. > c) If its too slow - and if profiling suggests that the spirit parser > is the bottleneck - then I would look at tweaking the grammar to speed > up parsing or replacing the spirit parser with a faster one. This is > my rule: "First make it work ASAP - then make it faster if necessary" > > But I already am somewhat familiar with spirit so it might not be an > interesting option for you. But then yo might be able to use the > current parser unchanged. Of course this would bring the huge benefit > that if the xml_archive parser is tweaked for some reason (there are a > couple of issues with special characters), you would automatically > inherit these changes and still be in sync. > > I made the choice to invest the effort to figure out spirit rather > than write my 10,000th file parser. Of course that was my decision and > may not be everyone's preference. > > > Good Luck > > Terence Wilson wrote: > > Robert, > > > > The utility I am writing needs to be able to extract a small portion > > from a large XML file generated by your library. Since it is > > performance sensitive I chose to use a SAX parser in order to avoid > > reading the whole file. Would it be much work to do this with the > > Spirit parser? > > > > As always, thanks for the super-fast response. > > > > Best regards, > > > > _______________________________________________ > Boost-users mailing list > Boost-users(a)lists.boost.org > http://lists.boost.org/mailman/listinfo.cgi/boost-users _______________________________________________ Boost-users mailing list Boost-users(a)lists.boost.org http://lists.boost.org/mailman/listinfo.cgi/boost-users

1 0