RE: [boost] Wave C++ Review Begins Today - February 7, 2005
-----Original Message----- From: boost-bounces@lists.boost.org [mailto:boost-bounces@lists.boost.org] On Behalf Of Doug Gregor Sent: woensdag 9 februari 2005 17:23 To: boost@lists.boost.org Subject: Re: [boost] Wave C++ Review Begins Today - February 7, 2005
...
Even dereferencing *last could be a bug, if it is past-the-end. For iterators referencing contiguous memory (vectors and pointers only, as you've said), one would need to write
scanner.last = scanner.first + std::distance(first, last);
True, but that's an partial fix. For now, I've fixed that bug too in my vector<char> wrapper but it's becoming complex. I'm afraid someone will have to write re2cpp.
I really have to say Dinkumware did a fine job here, this would have been very hard to find without these checks.
FWIW, STLport and newer versions of GCC (Apple GCC 3.3, FSF GCC 3.4) have similar debugging modes. They've been invaluable in rooting out subtle bugs like those that stem from iterator invalidation.
Yep, although I'm currently wondering if they're 100% right. Is it legal to compare iterators from different containers? I can't find the appropriate wording so I'll just trust Dinkumware here. My current problem is, somewhere deep down in cpp_macromap.hpp a simple innocent if (first != last) statement (line 433) triggers another assertion. These are iterator_facade's (which I don't grok) and end up comparing iterators to two distinct containers. Looking the other way on the callstack, I see some deeply suspect iterators (singular, destroyed, ?) Too messy to say what happened. Regards, Michiel Salters This e-mail and any attachment is for authorised use by the intended recipient(s) only. It may contain proprietary material, confidential information and/or be subject to legal privilege. It should not be copied, disclosed to, retained or used by, any other party. If you are not an intended recipient then please promptly delete this e-mail and any attachment and all copies and inform the sender. Thank you.
Michiel Salters wrote:
Even dereferencing *last could be a bug, if it is past-the-end. For iterators referencing contiguous memory (vectors and pointers only, as you've said), one would need to write
scanner.last = scanner.first + std::distance(first, last);
True, but that's an partial fix. For now, I've fixed that bug too in my vector<char> wrapper but it's becoming complex. I'm afraid someone will have to write re2cpp.
It's not a problem of the re2c tool, it's merely a problem of the interfacing code exposing the re2c generated lexer to the library. I'm completely aware of (some of) the related problems, but had no real reason to fix it yet, simply because it worked... (never touch a running system :-). I'll have to investigate this in the near future to clean it up finally.
My current problem is, somewhere deep down in cpp_macromap.hpp a simple innocent if (first != last) statement (line 433) triggers another assertion. These are iterator_facade's (which I don't grok) and end up comparing iterators to two distinct containers. Looking the other way on the callstack, I see some deeply suspect iterators (singular, destroyed, ?)
Since I'm still installing the vc8 on my spare machine here, I'm not able to reproduce it yet. Just give me some additional time. Could you please send me your input files, which lead to the problem you've observed?
Too messy to say what happened.
Very difficult to tell from the distance. Will try to reproduce and fix it asap. Could you get in direct contact with me to sort out the problems? Regards Hartmut
On Feb 9, 2005, at 12:29 PM, Michiel Salters wrote:
I really have to say Dinkumware did a fine job here, this would have been very hard to find without these checks.
FWIW, STLport and newer versions of GCC (Apple GCC 3.3, FSF GCC 3.4) have similar debugging modes. They've been invaluable in rooting out subtle bugs like those that stem from iterator invalidation.
Oops, I forgot to mention that CodeWarrior has a debug mode, too.
Yep, although I'm currently wondering if they're 100% right.
Most likely, yes. They tend to be written by library lawyers who directly translate from the standardese to the most picky, specific implementation that the standardese allows. The usual complaint about debug modes is that they're too strict... for instance, a bug report was filed against GCC's debug mode because it complains when a singular iterator is copy-constructed. The debug mode is right, even though I've never seen a singular iterator that couldn't safely be copied.
Is it legal to compare iterators from different containers? I can't find the appropriate wording so I'll just trust Dinkumware here.
No, it is not legal to compare iterators from different containers. Operations that require two iterators mandate that the iterators be part of the operation's "domain". Iterators from different containers can never be in the same domain.
My current problem is, somewhere deep down in cpp_macromap.hpp a simple innocent if (first != last) statement (line 433) triggers another assertion. These are iterator_facade's (which I don't grok) and end up comparing iterators to two distinct containers. Looking the other way on the callstack, I see some deeply suspect iterators (singular, destroyed, ?)
Yikes... I'll dig into it when I get a chance to write my Wave review. Doug
Doug Gregor <dgregor@cs.indiana.edu> writes:
No, it is not legal to compare iterators from different containers. Operations that require two iterators mandate that the iterators be part of the operation's "domain". Iterators from different containers can never be in the same domain.
I generally agree, but think it gets a bit murky when we deal with std::list iterators, since they can be spliced back and forth between different containers without being invalidated. It seems strange that we can compare the two iterators into a list and that's valid. Then splice one element into another list--the iterators have not changed whatsoever--but now comparing them is invalid. Finally, splice the remaining element from the first list into the second, so they're both in the other list now. Presto, comparing the same two iterators is valid again. -- Chris
On Feb 9, 2005, at 3:36 PM, Chris Uzdavinis wrote:
Doug Gregor <dgregor@cs.indiana.edu> writes:
No, it is not legal to compare iterators from different containers. Operations that require two iterators mandate that the iterators be part of the operation's "domain". Iterators from different containers can never be in the same domain.
I generally agree, but think it gets a bit murky when we deal with std::list iterators, since they can be spliced back and forth between different containers without being invalidated.
Splice essentially transfers ownership of iterations from one container to another...
It seems strange that we can compare the two iterators into a list and that's valid. Then splice one element into another list--the iterators have not changed whatsoever--but now comparing them is invalid.
Sure, but I can do this, too: vector<int> v1, v2; vector<int>::iterator i, j; i = v1.begin(); j = v1.end(); i == j; i = v2.begin(); j = v2.end(); i == j; The domain changed for both iterators, just like in the list example. Domains are not static properties, and this is _really_ important. It shows up in the stream iterators, too, where the specifications of != and == make absolutely no sense without careful considering the domain of the operations. Doug
Doug Gregor wrote:
My current problem is, somewhere deep down in cpp_macromap.hpp a simple innocent if (first != last) statement (line 433) triggers another assertion. These are iterator_facade's (which I don't grok) and end up comparing iterators to two distinct containers. Looking the other way on the callstack, I see some deeply suspect iterators (singular, destroyed, ?)
Yikes... I'll dig into it when I get a chance to write my Wave review.
Yikes indeed ;-) But rest assured, I've fixed the problems asserted on by the iterator test code of the VC8 STL (at least the problems reported by you and Michiel). Because these bug were quite serious, I've uploaded a new version of Wave, which you should use for further investigations here: http://spirit.sourceforge.net/dl_more/wave-1.1.13.zip, additionally I've updated the Boost-sandbox CVS to contain the very same bits. Thanks for your insights! Regards Hartmut
Michiel Salters wrote:
-----Original Message----- From: boost-bounces@lists.boost.org [mailto:boost-bounces@lists.boost.org] On Behalf Of Doug Gregor Sent: woensdag 9 februari 2005 17:23 To: boost@lists.boost.org Subject: Re: [boost] Wave C++ Review Begins Today - February 7, 2005
...
Even dereferencing *last could be a bug, if it is past-the-end. For iterators referencing contiguous memory (vectors and pointers only, as you've said), one would need to write
scanner.last = scanner.first + std::distance(first, last);
True, but that's an partial fix. For now, I've fixed that bug too in my vector<char> wrapper but it's becoming complex. I'm afraid someone will have to write re2cpp.
It is possible to use re2c generated code in a c++ context. However, to explain the current situation, the original re2c c++ lexer was actually written in pure C (thus the origin of the aq_* code), and it looks like Hartmut has only changed the minimum necessary to integrate it with Wave. Like he said, if it works, don't mess with it :-) I originally wrote the re2c c++ scanner as a benchmarking tool to compare performance of re2c/flex/SLex generated scanners. At the time re2c won by about a factor of 2 IIRC. I did it in C because my #1 goal was performance. I found it interesting that, using the exact same source, if I compiled it as C instead of C++, it was about 10% faster. But that was years ago with gcc 2.95.2, who knows how things would stand today? -- Dan Nuffer
Dan Nuffer wrote:
It is possible to use re2c generated code in a c++ context. However, to explain the current situation, the original re2c c++ lexer was actually written in pure C (thus the origin of the aq_* code), and it looks like Hartmut has only changed the minimum necessary to integrate it with Wave. Like he said, if it works, don't mess with it :-)
Yes, I've written minimal interfacing code only. Nevertheless I had to change the lexer itself (mainly the token regex's) quite a bit.
I originally wrote the re2c c++ scanner as a benchmarking tool to compare performance of re2c/flex/SLex generated scanners. At the time re2c won by about a factor of 2 IIRC. I did it in C because my #1 goal was performance. I found it interesting that, using the exact same source, if I compiled it as C instead of C++, it was about 10% faster. But that was years ago with gcc 2.95.2, who knows how things would stand today?
I haven't obversed such a runtime difference during my tests, but what I do have observed is a significant compile time difference in about factor 10 and more (between the compilation of the re2c generated giant switch statement with a C and a C++ compiler). Regards Hartmut
participants (6)
-
Chris Uzdavinis -
Dan Nuffer -
Doug Gregor -
Douglas Gregor
-
Hartmut Kaiser -
Michiel Salters