I'm finding an increasing need for an in-file memory log: specifically when my program asserts I want to see where it has been. There are two problems with normal logging, i.e. appending to a disk file: * The files get very, very big (I've filled my home partition before) * All that writing to disk slows the application down noticeably. I checked log4cpp and John Torjo's logging library and neither seem to offer this, but it seems it could fit as one of the logging targets. I had in mind an object that is a wrapper for e.g. a 1Mb char buffer with the same interface as std::ostream, so I can just write: log_buffer<<"In func1, x="<<x<<", obj="<<obj<<"\n"; And then when my program asserts I can pass log_buffer as a parameter to be output (requires it to have operator<< defined): SMART_ASSERT(y!=1)(y)(log_buffer); Does anyone know of anything along these lines? Does anyone have the same need, or just me? I found some circular buffers that nearly do what I want [1] [2] [3], but they are lower-level as they don't offer the ostream interface. (In my case I don't need thread-safety.) Darren [1]: http://www.codeguru.com/Cpp/Cpp/cpp_mfc/arrays/article.php/c4013/ [2]: http://www.codeguru.com/Cpp/Cpp/cpp_mfc/buffers/article.php/c859/ [3]: Thread-safe circular queue, CUJ, June 2004.
"Darren Cook" wrote:
I found some circular buffers that nearly do what I want [1] [2] [3], but they are lower-level as they don't offer the ostream interface. (In my case I don't need thread-safety.)
Did you look on Boost circular_buffer (currently in Files section)? It is not yet oficially released but should get into 1.33. /Pavel
Pavel Vozenilek wrote:
"Darren Cook" wrote:
I found some circular buffers that nearly do what I want [1] [2] [3], but they are lower-level as they don't offer the ostream interface. (In my case I don't need thread-safety.)
Did you look on Boost circular_buffer (currently in Files section)? It is not yet oficially released but should get into 1.33.
/Pavel
You may find cycle_iterator useful. I have uploaded to files area: 1. cycle_iterator.hpp 2. cycle_iterator2.hpp 3. Ring2.H 1. cyclic iterator adaptor. Keeps track of wraparounds. 2. similar to 1, but doesn't track wraparounds. This is more efficient in case you don't need that feature. 3. Ring adapts a container making it cyclic.
Did you look on Boost circular_buffer (currently in Files section)? It is not yet oficially released but should get into 1.33.
You may find cycle_iterator useful. I have uploaded to files area:
I had a look but wasn't sure where it fits in - is it an alternative to boost::circular_buffer? Darren
Darren Cook wrote:
Did you look on Boost circular_buffer (currently in Files section)? It is not yet oficially released but should get into 1.33.
You may find cycle_iterator useful. I have uploaded to files area:
I had a look but wasn't sure where it fits in - is it an alternative to boost::circular_buffer?
Yes. The behaviour of circular_buffer and cycle_iterator are not the same. I grabbed the latest circular_buffer.zip from FILES, but there is no doc. AFAICT, the main difference is that cycle_iterator is meant to be a fixed size, while I believe circular_buffer is variable size up to a maximum? cycle_iterator (and Ring) are also smaller and simpler code.
You may find cycle_iterator useful. I have uploaded to files area:
I had a look but wasn't sure where it fits in - is it an alternative to boost::circular_buffer?
Yes. The behaviour of circular_buffer and cycle_iterator are not the same. I grabbed the latest circular_buffer.zip from FILES, but there is no doc.
The docs start at circular_buffer/libs/circular_buffer/index.html. The file I downloaded was circular_buffer_v3.6.zip.
AFAICT, the main difference is that cycle_iterator is meant to be a fixed size, while I believe circular_buffer is variable size up to a maximum? cycle_iterator (and Ring) are also smaller and simpler code.
I think circular_buffer can be either fixed or variable size, but I've only glanced at the docs so far so I'll leave it to the author to discuss pros and cons with you :-). Darren
Did you look on Boost circular_buffer (currently in Files section)? It is not yet oficially released but should get into 1.33.
Since we have branched for release, now would be the perfect time to check it circular buffer into the main line and start integrating it into boost -- so we aren't rushing it into the next release... Jeff
Darren Cook wrote:
I'm finding an increasing need for an in-file memory log: specifically when my program asserts I want to see where it has been.
There are two problems with normal logging, i.e. appending to a disk file: * The files get very, very big (I've filled my home partition before) * All that writing to disk slows the application down noticeably.
I checked log4cpp and John Torjo's logging library and neither seem to
not yet, but in the near future I want to add cyclic logs (or rotating logs, if you wish). Note: You can use ts_appender() (formely known as ts_write()), and thus the writing will be on a dedicated thread (I've used this in a real-life app. - it's *very* fast). Just for fun, something like: // assume you've BOOST_DEFINE_LOG(gui,"app.gui") somewhere add_appender("app.gui", ts_appender(write_to_file("out.txt")) ); // written to file in its own dedicated thread - very fast BOOST_LOG(gui) << "whatever" << " you " << "want"; But for asserts, this is not useful :(
offer this, but it seems it could fit as one of the logging targets. I had in mind an object that is a wrapper for e.g. a 1Mb char buffer with the same interface as std::ostream, so I can just write: log_buffer<<"In func1, x="<<x<<", obj="<<obj<<"\n";
And then when my program asserts I can pass log_buffer as a parameter to be output (requires it to have operator<< defined): SMART_ASSERT(y!=1)(y)(log_buffer);
As a side-note, I'm planning to update the SMART_ASSERT lib in about 3 weeks ;) At that time, I want to make it interoperate with Boost.Log . Thus, you should have your failed asserts in no time ;) However, the fact you want the circular log buffer makes me think that you have quite a few failed assertions ;) Anyway, I'll try to improve SMART_ASSERT in the near future. (now that I think about it, maybe it should use Boost.Log for its logging completely...) Best, John -- John Torjo, Contributing editor, C/C++ Users Journal -- "Win32 GUI Generics" -- generics & GUI do mix, after all -- http://www.torjo.com/win32gui/ -- v1.5 - tooltips at your fingertips (work for menus too!) + bitmap buttons (work for MessageBox too!) + tab dialogs, hyper links, lite html
not yet, but in the near future I want to add cyclic logs (or rotating logs, if you wish).
That would help in the cases where I need the log on disk. As you said in another mail building an in-memory circular log on top of boost::circular_buffer should not be too hard, so I'll try.
As a side-note, I'm planning to update the SMART_ASSERT lib in about 3 weeks ;)
That is good news - I've found it very useful in tracking down bugs. All it lacks is stack trace (which is simply not possible I believe) and a history log (that I'm talking about here).
However, the fact you want the circular log buffer makes me think that you have quite a few failed assertions ;)
Probably about one line in 4 is an assert (mostly preconditions at the start of each function). I'm hoping Thorsten Ottosen's contract programming will make it into the standard so the compiler will understand asserts and be able to optimize away the redundant ones.
Anyway, I'll try to improve SMART_ASSERT in the near future. (now that I think about it, maybe it should use Boost.Log for its logging completely...)
Only if it stays behind the scenes - I like that I define its behaviour once in main(); I wouldn't want to have to specify a log name in every SMART_ASSERT() line. Darren
Anyway, I'll try to improve SMART_ASSERT in the near future. (now that I think about it, maybe it should use Boost.Log for its logging completely...)
Only if it stays behind the scenes - I like that I define its behaviour once in main(); I wouldn't want to have to specify a log name in every SMART_ASSERT() line.
Behind the scenes it'll be ;) Best, John -- John Torjo, Contributing editor, C/C++ Users Journal -- "Win32 GUI Generics" -- generics & GUI do mix, after all -- http://www.torjo.com/win32gui/ -- v1.5 - tooltips at your fingertips (work for menus too!) + bitmap buttons (work for MessageBox too!) + tab dialogs, hyper links, lite html
Darren Cook <darren@dcook.org> writes:
I'm hoping Thorsten Ottosen's contract programming will make it into the standard so the compiler will understand asserts and be able to optimize away the redundant ones.
Compilers are already allowed to do that, and I predict no proposal that requires it to be done will ever be accepted into the standard. -- Dave Abrahams Boost Consulting http://www.boost-consulting.com
I'm hoping Thorsten Ottosen's contract programming will make it into the standard so the compiler will understand asserts and be able to optimize away the redundant ones.
Compilers are already allowed to do that, and I predict no proposal that requires it to be done will ever be accepted into the standard.
"be able" falls somewhere between "allowed to" and "required to". An optimizer will have a hard job analyzing all the code in a function so see if any can be skipped. But if it can see a precondition is always satisfied then it can jump directly to the body of the function and skip the precondition. E.g. void f(int x){ precondition{ x>=0 && x<=99; } //1 ... //2 } f(atoi(argv[2])); //Jumps to f, at 1 f(34); //Jumps to f, at 2 void g(int z){ precondition{ z>=10 && z<50; } f(z*2); //Can also jump to f at 2, as range is 20..98 } I don't think Thorsten's proposal requires optimizations like this. But I don't know enough about compiler writing to know if the above is ever likely to be implemented. Darren
Darren Cook <darren@dcook.org> writes:
I'm hoping Thorsten Ottosen's contract programming will make it into the standard so the compiler will understand asserts and be able to optimize away the redundant ones. Compilers are already allowed to do that, and I predict no proposal that requires it to be done will ever be accepted into the standard.
"be able" falls somewhere between "allowed to" and "required to". An optimizer will have a hard job analyzing all the code in a function so see if any can be skipped. But if it can see a precondition is always satisfied then it can jump directly to the body of the function and skip the precondition.
E.g. void f(int x){ precondition{ x>=0 && x<=99; } //1 ... //2 }
f(atoi(argv[2])); //Jumps to f, at 1 f(34); //Jumps to f, at 2
If f is inlined, I'm positive that compilers today can do that.
void g(int z){ precondition{ z>=10 && z<50; } f(z*2); //Can also jump to f at 2, as range is 20..98 }
And I'm pretty sure that compilers today can do that one.
I don't think Thorsten's proposal requires optimizations like this. But I don't know enough about compiler writing to know if the above is ever likely to be implemented.
It is just as implementable for assert as it is with Thorsten's proposal. -- Dave Abrahams Boost Consulting http://www.boost-consulting.com
participants (6)
-
Darren Cook -
David Abrahams -
Jeff Garland -
John Torjo -
Neal D. Becker -
Pavel Vozenilek