Re: [Boost-users] yet another logging library
Hi Mojmir, I think you would need to explain the advantages of your (future) library over existing logging libraries. In what circumstances would your library be the better choice? Best regards, Maarten ----- Original Message ---- From: Mojmir Svoboda <mojmir.svoboda@illusionsoftworks.com> To: boost-users@lists.boost.org Sent: Wednesday, June 4, 2008 5:07:14 PM Subject: [Boost-users] yet another logging library Hello, i'm trying to create a logging library framework. It is my second attempt, the first can be seen at http://www.codeplex.com/flog . There is some documentation in the doc directory that is perhaps worth reading. The first version used boost::iostreams and not very smart locking mechanism in mt environment (in terms of performance mainly), but served more as a verification 'it can be done and it is not that ugly as i thought' than a real library candidate. Now i'd like to focus on performance and code size, so I made a sketch of the next version. It is really small and is rather a model than anything usable, but before I start implementation, I'd like to know opinion of library writers and users - what do you think about the concept? is it worth continuing, or i'd better go doing something better? :) The sketch of the next version can be seen at http://www.codeplex.com/fflog It's meant to be illustration of the interface and it's usage itself. Short summary of the design: User specifies compile-time vector of requested filters that apply to the logged text. Each filter requiring user's input is tagged and when logging function is invoked, the supplied types are checked against expected type, set to filter and later written to a buffer. Buffers are managed by the sink_manager and likely is one of few places that has to be mt-safe, as the rest is on the stack. I thought about some lock-free queues of buffers that are assigned to caller and filled returned back to write. The actual write of the buffer can be performed in dedicated lower-priority thread. Run-time filtering like level or context is possible, but not implemented yet. The main idea can be seen in the 1st version in runtime_filtering.hpp. Again, runtime values are shared among threads, so they have to be protected in mt environment. Many thanks for your attention, Mojmir _______________________________________________ Boost-users mailing list Boost-users@lists.boost.org http://lists.boost.org/mailman/listinfo.cgi/boost-users
Morning, * Maarten Nieber <hallomaarten@yahoo.com> [2008-06-05 03:57:09 -0700]:
I think you would need to explain the advantages of your (future) library over existing logging libraries. In what circumstances would your library be the better choice?
first version was a bloated mess, the second should be small and fast. Mainly after reviewing ACE and log4c++. John's work seems to be better in this aspect - i took a quick look fastest_no_ostr_like.cpp comparing to John's work i'd like to avoid heap allocations, exceptions and rtti. also i'd like to avoid mutexes, because i consider it a very heavy hammer. but of course the choice is up to user at the end. there is a dark side of my approach: the user may not pay at the runtime, but certainly he'll suffer at compile-time. what you gain is control of the types passed to logging framework (how many times we all messed a printf's format and arguments? :) Regards, Mojmir
On Mon, Jun 09, 2008 at 01:46:51PM +0200, Mojmir Svoboda wrote:
there is a dark side of my approach: the user may not pay at the runtime, but certainly he'll suffer at compile-time. what you gain is control of the types passed to logging framework (how many times we all messed a printf's format and arguments? :)
<0.02> I would not consider this a dark side. Enduring long compile cycles seems tenable when optimizing for runtime-performance. Your design sounds appealing to me. I am exactly looking for a small and efficient logging library that provides basic functionality (levels, filters, facilities, filters and alike) and has been designed from scratch to be used in multi-threaded programs. </0.02> Matthias -- Matthias Vallentin vallentin@icsi.berkeley.edu pgp/gpg: 0x37F34C16
Matthias Vallentin wrote:
On Mon, Jun 09, 2008 at 01:46:51PM +0200, Mojmir Svoboda wrote:
there is a dark side of my approach: the user may not pay at the runtime, but certainly he'll suffer at compile-time. what you gain is control of the types passed to logging framework (how many times we all messed a printf's format and arguments? :)
<0.02> I would not consider this a dark side. Enduring long compile cycles seems tenable when optimizing for runtime-performance. Your design sounds appealing to me. I am exactly looking for a small and efficient logging library that provides basic functionality (levels, filters, facilities, filters and alike) and has been designed from scratch to be used in multi-threaded programs. </0.02>
At this time, my library already does this - you can certainly specify whether you want thread-safety or not, and the lib is certainly efficient. Best, John -- http://John.Torjo.com -- C++ expert http://blog.torjo.com ... call me only if you want things done right
hello, John :)
At this time, my library already does this - you can certainly specify
going quickly through your code i definitly must admit you are farer than i am. there's no apparent reason start new thingy. for example i'd probably never mess with internationalization ;)
whether you want thread-safety or not, and the lib is certainly efficient.
can you give me some hints in the right direction, please? i mean some general advices "use this if you want be small, this if you want be fast, avoid that at all costs etc". i'd like both, of course. i started with your sample and got it down to 12kB (assume 5k is C helloworld). that's not quite bad, i suppose, but still twice as it should be ;-P there are still 9 allocation - there is some vector at initialization time, but i'd have to read the sources more carefully. perhaps you could point out these places where expensive and other interesting things pop up. minor issues i found: - logger.hpp:248 when cache is off, source cannot be compiled (missing ifdef probably) - pthreads are detected incorrecty. even in singlethr model mutex is required. - it would be nice if it can compile with -fno-rtti -fno-exceptions (op_equal and shared_ptr) see *) *) note that it would be nice that all boost code that does not really need rtti could switch it off (fedora7, boost-1_35_0, gcc-4.1.2, bjam not used) best regards, Mojmir ======8<============== #define BOOST_LOG_BEFORE_INIT_IGNORE_BEFORE_INIT 1 #include <cstdio> #include <boost/logging/logging.hpp> #include <boost/logging/format.hpp> using namespace boost::logging; struct no_gather { typedef const char* msg_type ; const char * m_msg; no_gather() : m_msg(0) {} const char * msg() const { return m_msg; } void *out(const char* msg) { m_msg = msg; return this; } }; struct printf_t : destination::is_generic { void operator() (char const * msg) const { printf("%s\n", msg); } }; typedef logger<no_gather, printf_t> app_log_type; filter::no_ts * g_log_filter () { static filter::no_ts l; return &l; } app_log_type* g_log_app () { static app_log_type l; return &l; } int main () { g_log_app()->mark_as_initialized(); get_logger_base(g_log_app())->read_msg().gather().out("hello world\n"); } ======8<==============
Mojmir Svoboda wrote:
hello, John :)
At this time, my library already does this - you can certainly specify
going quickly through your code i definitly must admit you are farer than i am. there's no apparent reason start new thingy. for example i'd probably never mess with internationalization ;)
It's your call ;) I think I've done about 6 other versions until I got to this one - which I will change as well ;)
whether you want thread-safety or not, and the lib is certainly efficient.
can you give me some hints in the right direction, please? i mean some general advices "use this if you want be small, this if you want be fast, avoid that at all costs etc". i'd like both, of course.
Well, this is quite hard: - the locking strategy - make it a policy, which you can change later; this will also make it easy for you to test, using a single threaded app - why do you apply filters to the logged text? A filter should be something able to say "yes" or "no" without looking at the text. First of all, if the filter says "no", you shouldn't need to do any processing of the text. - I believe having a compile time vector for formatting the output is a mistake: typedef vector<time, sep, rt_level, sep, file, sep, line<>, sep, fn, sep, msg>::type pipe_mania; The user should be able to specify that at runtime - what if he wants to configure the log at runtime? Other than that, I need to see more docs about what you're going to implement.
i started with your sample and got it down to 12kB (assume 5k is C helloworld). that's not quite bad, i suppose, but still twice as it should be ;-P
Oh well, I don't care that much about size nowadays.
there are still 9 allocation - there is some vector at initialization time, but i'd have to read the sources more carefully. perhaps you could point out these places where expensive and other interesting things pop up.
minor issues i found: - logger.hpp:248 when cache is off, source cannot be compiled (missing ifdef probably)
Ouch. My bad - anyway, the caching will be gone in v3 of the lib ;)
- pthreads are detected incorrecty. even in singlethr model mutex is required.
What do you mean?
- it would be nice if it can compile with -fno-rtti -fno-exceptions (op_equal and shared_ptr) see *)
*) note that it would be nice that all boost code that does not really need rtti could switch it off
Yes I agree - I want to add this in the future ;) Best, John -- http://John.Torjo.com -- C++ expert http://blog.torjo.com ... call me only if you want things done right
hello John, * John Torjo <john.groups@torjo.com> [2008-06-12 12:02:53 +0300]:
can you give me some hints in the right direction, please? i mean some general advices "use this if you want be small, this if you want be fast, avoid that at all costs etc". i'd like both, of course.
Well, this is quite hard: - the locking strategy - make it a policy, which you can change later; this will also make it easy for you to test, using a single threaded app
that is necessary nowadays. i still had no time to do my homework and explore mt in your code... i will until end of the week.
- why do you apply filters to the logged text? A filter should be something able to say "yes" or "no" without looking at the text. First of all, if the filter says "no", you shouldn't need to do any processing of the text.
if you are talking about my approach of filters, i mixed three concepts according what the user has to do: 1. nothing at all (filter simply appends text, for example timestamp) 2. supply an argument - for filters like file, line taking __FILE__ etc 3. 2. + supply comparation logic and run-time value. this is case of levelling logged text and filtering out messages > certain level. it depends on needs of user and is specified fully by the mpl::vector. the run-time values of course requires shared access among threads, so protecting resources has to be done carefully. ordinary mutex is too much, spinlock fits better.
- I believe having a compile time vector for formatting the output is a mistake: typedef vector<time, sep, rt_level, sep, file, sep, line<>, sep, fn, sep, msg>::type pipe_mania; The user should be able to specify that at runtime - what if he wants to configure the log at runtime?
no he will _NOT_, that is the beauty ;)) if he still do, let him use log4jpp. according to my (i admit that short perhaps) experiences, this is very rarely needed. you can either modify source and recompile or use log4j++ like i said. for really simple filtering you can always use levels, context (i call that some "logic areas" of code like data flow, system message passing, etc), threadids, etc. ie simple cases already built in the chain at the compile-time. the run-time filtering logging facility should be really stupid and fast. no regexps etc. but more refined filtering for human analysis can be done by some external tool and that is where i aim to: 1. you configure your logging and you don't really pay what you dont use 2. you compile your application 3. run it 4. (optional) configure your run-time filtering values like level 5. stop the program and grab the text 6. analyse the text in some tool this tool colorizes thread ids, perform some regex replacements, ...whatever in my case it's mighty AWK, dog bless that thingy.
Other than that, I need to see more docs about what you're going to implement.
basic logging facilities. logger, some accessor (singleton?), simple appenders, few run-time filters and few sinks (file/socket).
Oh well, I don't care that much about size nowadays.
oh my goth, aren't you one of THEM? :) i'm thinking myself lately that this is kind of black plague of c++...
- pthreads are detected incorrecty. even in singlethr model mutex is required. What do you mean?
i did not use bjam for the build as i really think this tool surely comes from hell. my line is simple: g++ -I ~/devel/boost -I ~/devel/logging/ aa1.cpp but your code incorrectly detects presence of -pthread (/MT on msvc i think) switch and includes some code using pthreads even if -pthread is not present. i don't remember exactly, but $BOOST_ROOT/boost/config/...stuff.. makefiles do that for you and sets BOOST_HAS_THREADS correctly. perhaps i am wrong and it is bjam who is doing that. best regards, mojmir
Mojmir Svoboda wrote:
The user should be able to specify that at runtime - what if he wants to configure the log at runtime?
no he will _NOT_, that is the beauty ;)) if he still do, let him use log4jpp. according to my (i admit that short perhaps) experiences, this is very rarely needed. you can either modify source and recompile or use log4j++ like i said.
My experience is with large products with many subsystems, worked on by many developers over the course of years. The most effective logging machinery: - Allows you to turn on logging at runtime, so your support person in e-mail dialog with an irritated user can say, "Okay, please set this configuration switch and try that again." Some bugs can only be reproduced on a user's machine. On the other hand, you don't want to generate large log files for every user, every run. - Allows you to be selective about what logging output is produced, because if you turn on EVERYTHING, your user has trouble even storing and mailing you the resulting file -- never mind your problems trying to discover what might be relevant. (This is the result of accumulating helpful logging output in many interacting subsystems over the course of years.) - Has trivial runtime cost for a suppressed log message. In particular, no string allocation or formatting is performed. Ideally, the amortized runtime cost is nothing more than examining a static bool and a jump. There are probably people for whom compile-time controls suffice. But such a library wouldn't be useful to our developers.
Nat Goodspeed Wrote:
- Allows you to turn on logging at runtime
- Allows you to be selective about what logging output is produced
- Has trivial runtime cost for a suppressed log message.
Just to back this up Mojmir, these are pretty much essential for a useful logging system really. As a developer I can step through my code to find out what's happening. Logging can be useful while developing but the only reason I include all the extra code in my apps is to allow my company to provide support once the system has gone live. If it didn't do that I'd stick to the debugger and if the live system is running correctly then you don't want the overhead...
* Patrick Loney <Patrick.Loney@InteractiveTS.com> [2008-06-12 15:48:42 +0100]:
Nat Goodspeed Wrote:
- Allows you to turn on logging at runtime
- Allows you to be selective about what logging output is produced
- Has trivial runtime cost for a suppressed log message.
Just to back this up Mojmir, these are pretty much essential for a useful logging system really. As a developer I can step through my code to find out what's happening. Logging can be useful while developing but the only reason I include all the extra code in my apps is to allow my company to provide support once the system has gone live. If it didn't do that I'd stick to the debugger and if the live system is running correctly then you don't want the overhead...
see my response for Nat. i admit log is a saviour frequently. saved my ass so many times! and even if it does not pinpoint the problem accurately, it provides at least probable direction of the problem. and knowing your log means knowing your system. this always helps. there are drawbacks although: - it takes resources. okay that's the price you pay but i do not want pay more that it deserves. - serializes multithreaded code there are sometimes faults that occur only with logging off, because of the serialization by logging library. theser are very nasty :) on the other hand it may be solved by removing all shared rsrcs, perhaps by saving each thread's log to separate sink? regards, mojmir
Mojmir Svoboda skrev:
- serializes multithreaded code there are sometimes faults that occur only with logging off, because of the serialization by logging library. theser are very nasty :) on the other hand it may be solved by removing all shared rsrcs, perhaps by saving each thread's log to separate sink?
I designed a logging library a while ago that we're using quite extensively in a performance critical multi-threaded environment, and it actually does precisely that. Each thread has its own logger which can be connected to a "sink" specific for that thread. This approach was chosen after making some observations: 1) Synchronization has a performance penalty. 2) Synchronization might affect the behaviour of the program when increasing the amount of logged data, potentially leading to heisenbugs. 3) Sometimes you actually never need to merge the streams of events. 4) Since the logged data is used for "external" diagnostics only, the library should allow the application generating the logs to do as little as necessary before storing them away. Storing primary measurement data, in a sense. The idea was to separate the generation of a log event from the synchronization, transformation and storage of log events. In this way, the cost of synchronization of multiple log streams is moved to a later stage. It can either be done in a designated "merger" thread inside the application, reading from lock-free queues connected to the "thread loggers", or each stream can be sent to a different file or network socket and merged off-line by a diagnostics tool (if needed). Of course, in the latter case you might get synchronization inside the standard libraries or the OS instead. On the other hand, this can easily be amortized, albeit with the risk of loosing unflushed log data in the event of a segmentation fault. Best regards, Josef
... be connected to a "sink" specific for that thread.
This approach was chosen after making some observations: 1) Synchronization has a performance penalty. and not a trivial one :/
2) Synchronization might affect the behaviour of the program when increasing the amount of logged data, potentially leading to heisenbugs. heisenbugs ;)) 3) Sometimes you actually never need to merge the streams of events. 4) Since the logged data is used for "external" diagnostics only, the library should allow the application generating the logs to do as little as necessary before storing them away. Storing primary measurement data, in a sense.
i agree there. i'd like to minimize the interference of the debugging mode from the normal flow as much as possible. maybe with exception of the runtime filtering facilities. i thought about it and these runtime values are read only most of the time. it depends - use either spinlocks if they are likely to be modified at runtime or marked simply as volatile and restrict write only to initialization phase.
The idea was to separate the generation of a log event from the synchronization, transformation and storage of log events.
exactly.
In this way, the cost of synchronization of multiple log streams is moved to a later stage. It can either be done in a designated "merger" thread inside the application, reading from lock-free queues connected to the "thread loggers", or each stream can be sent to a different file or network socket and merged off-line by a diagnostics tool (if needed).
aaaaah, the agony of re-discovering the already discovered ;-) don't you have sources available? best regards, mojmir
* Nat Goodspeed <nat@lindenlab.com> [2008-06-12 10:23:28 -0400]:
Mojmir Svoboda wrote:
no he will _NOT_, that is the beauty ;)) if he still do, let him use log4jpp. according to my (i admit that short perhaps) experiences, this is very rarely needed. you can either modify source and recompile or use log4j++ like i said.
My experience is with large products with many subsystems, worked on by many developers over the course of years.
hmm, i've been few years in telco, so i have the picture. blurred may that one be ;)
The most effective logging machinery:
- Allows you to turn on logging at runtime, so your support person in
of course, that is necessary.
- Has trivial runtime cost for a suppressed log message... runtime cost is nothing more than examining a static bool and a jump.
this is related to 1st and i second that.
- Allows you to be selective about what logging output is produced, because if you turn on EVERYTHING, your user has trouble even storing and mailing you the resulting file -- never mind your problems trying to discover what might be relevant. (This is the result of accumulating helpful logging output in many interacting subsystems over the course of years.)
i know, logs can be really huge. but i never meant to not provide what you call runtime filtering. i perhaps said something badly. i thought you wanted add another filter at runtime. what i am saying is: User is not allowed to change format of the logged text at runtime, by changing the formatting chain. i know it can be done, but i don't want to as i consider it unnecessary. the reason why is that as you have large codebase with lot of outputs, your people probably have tools helping them analyse the logs: regexps, indents, scripts, filters to gnuplot etc etc. if you change order of the logged columns, your tools are confused. my approach is simply: think before picking format and then let it be the same forever (or at least for some long period). as a library user you can change it during development of course, but to take place, you have to recompile. i hope that is clear, now and sorry for confusion i made.
There are probably people for whom compile-time controls suffice. But such a library wouldn't be useful to our developers.
see above: compile-time approach fixes only the order of filters, it does not disallow runtime filtering. many thanks for feedback, mojmir
Mojmir Svoboda schrieb:
i know, logs can be really huge.
but i never meant to not provide what you call runtime filtering. i perhaps said something badly. i thought you wanted add another filter at runtime.
what i am saying is: User is not allowed to change format of the logged text at runtime, by changing the formatting chain. i know it can be done, but i don't want to as i consider it unnecessary.
the reason why is that as you have large codebase with lot of outputs, your people probably have tools helping them analyse the logs: regexps, indents, scripts, filters to gnuplot etc etc. if you change order of the logged columns, your tools are confused.
my approach is simply: think before picking format and then let it be the same forever (or at least for some long period). as a library user you can change it during development of course, but to take place, you have to recompile.
I think also that it is not so important to change the format during runtime. It is a nice to have feature but at least for me not a must. If you can configure the output format it has one big advantage: you can change the format of older products without recompiling the program. The most important thing for me that it does not cost a lot of time to format the message.
i hope that is clear, now and sorry for confusion i made.
best regards Hansjörg
On Fri, Jun 13, 2008 at 09:20:43AM +0200, Mojmir Svoboda wrote:
but i never meant to not provide what you call runtime filtering. i perhaps said something badly. i thought you wanted add another filter at runtime.
Did anybody of you look at how Solaris TNF framework or DTrace framework are implemented? A lot can be accomplished by tricks at link stage, and DTrace with UDT provider offers even more (selective filtering and data aggregation at run-time) than I could ask of any logging library. And that at a negligible cost for disabled log points. [No, I'm not proposing to reimplement all of DTrace. But what you're describing already exists, so it might be worth looking into how these things are implemented. Or, if nothing else, write a logging back-end for DTrace and/or TNF.]
good morning,
Did anybody of you look at how Solaris TNF framework or DTrace framework are implemented? A lot can be accomplished by tricks at link stage, and DTrace with UDT provider offers even more (selective filtering and data aggregation at run-time) than I could ask of any logging library. And that at a negligible cost for disabled log points.
just finished the usenix doc so that i got the picture. wow, i am quite astonished, i must say. this seems like every developper's dream... i worked with 5.8 and then quite lost a track. i used truss, strace, valgrind, purify, etc for achieve part of that goal, but apparently dtrace is lot more.
[No, I'm not proposing to reimplement all of DTrace. But what you're it would be perhaps easier to port it than rewrite, anyway
describing already exists, so it might be worth looking into how these things are implemented. Or, if nothing else, write a logging back-end for DTrace and/or TNF.
i'll certainly have a look at it, thanks! best regards, mojmir
* Zeljko Vrba <zvrba@ifi.uio.no> [2008-06-13 09:46:53 +0200]:
Did anybody of you look at how Solaris TNF framework or DTrace framework
good news everyone! :) port of dtrace for linux http://www.crisp.demon.co.uk/tools.html m.
Hi Mojmir,
- why do you apply filters to the logged text? A filter should be something able to say "yes" or "no" without looking at the text. First of all, if the filter says "no", you shouldn't need to do any processing of the text.
if you are talking about my approach of filters, i mixed three concepts according what the user has to do: 1. nothing at all (filter simply appends text, for example timestamp) 2. supply an argument - for filters like file, line taking __FILE__ etc 3. 2. + supply comparation logic and run-time value. this is case of levelling logged text and filtering out messages > certain level.
I'm still not sure why you need to text for that ;)
- I believe having a compile time vector for formatting the output is a mistake: typedef vector<time, sep, rt_level, sep, file, sep, line<>, sep, fn, sep, msg>::type pipe_mania; The user should be able to specify that at runtime - what if he wants to configure the log at runtime?
no he will _NOT_, that is the beauty ;)) if he still do, let him use log4jpp. according to my (i admit that short perhaps) experiences, this is very rarely needed.
Nat and Patrick already said it - run-time configuration is a must.
- pthreads are detected incorrecty. even in singlethr model mutex is required.
What do you mean?
i did not use bjam for the build as i really think this tool surely comes from hell.
I second that :) But it's good, every now and then ;) Best, John -- http://John.Torjo.com -- C++ expert http://blog.torjo.com ... call me only if you want things done right
I'm still not sure why you need to text for that ;)
sorry i don't understand what do you mean. i'm not a native speaker. did i explain something else you wanted? :)
Nat and Patrick already said it - run-time configuration is a must.
perhaps i cleared that out. run-time configuration like setting desired level is ok and already integrated in the first version of the library. changing order is not and probably never will be. have a nice day, mojmir
Mojmir Svoboda wrote:
run-time configuration like setting desired level is ok and already integrated in the first version of the library.
Going further out on a limb, and again speaking only for myself, I've always found that logging "level" doesn't map well to my needs. I typically need to see full detail for some two or three interacting subsystems, without any of the log output from any other subsystems. In homebrew logging systems, I usually associate a string tag with each log statement. The string tag is looked up on first reaching the statement, and every subsequent visit tests a cached bool. When the logging system initializes, it reads config info (from a file or registry data) to enable/disable particular string tags. I usually include the ability to express the configuration data as "only these tags" or "all but these tags." You can get fancier still by defining relationships between string tags, expressing them as hierarchies... but I don't consider that essential. The essence of the idea is the ability to control particular groups of related log messages. In my experience, the traditional "level" approach can be restated as: - hardly any useful information - not enough useful information - too much information - WAY too much information
* Nat Goodspeed <nat@lindenlab.com> [2008-06-13 09:25:54 -0400]:
I typically need to see full detail for some two or three interacting subsystems, without any of the log output from any other subsystems.
i'm using another orthogonal filtering facility - context. for example i want to dump only control messages exchanged between components of the program.
In homebrew logging systems, I usually associate a string tag with each log statement. The string tag is looked up on first reaching the statement, and every subsequent visit tests a cached bool.
When the logging system initializes, it reads config info (from a file or registry data) to enable/disable particular string tags. I usually
could you show me perhaps some of your config files? just for inspiration... how do you force reload of config? on unix i'd use a signal, but to be honest, have no big clue about windows...
You can get fancier still by defining relationships between string tags, it' always easier to blob up the design ;)
Going further out on a limb, and again speaking only for myself, I've always found that logging "level" doesn't map well to my needs.
- hardly any useful information - not enough useful information - too much information - WAY too much information
i admit you have point there. i was in situations where simple level was not enough. that's why i planned to add another run-time filtering facility; for example you could define your facilities like that: namespace flog { unsigned const bit_all = bin<11111111>::value; unsigned const bit_trace = bin<00000001>::value; unsigned const bit_hedumps = bin<00000010>::value; unsigned const bit_dataflow = bin<00000100>::value; ... unsigned const bit_synchro = bin<10000000>::value; /**@class default_filters * @brief sequence of default filters used for text formatting */ typedef runtime_flt<context, and_op > rt_context; typedef runtime_flt<level, std::less_equal> rt_level; typedef mpl::vector<rt_context ,I, rt_level ,I, time ,I, file,chr<':'>,line, I, msg>::type default_filters; } // namespace flog .. and then somthing like: LOG(l, bit_debug, 7, "will enable trace..."); runtime_set(l, identity<rt_context>(), bit_hexdump | bit_trace); { ENTRY(l, 7, "void foo(T) [with T = long int]"); } i'd really like to know your opinion on that. best regards, mojmir
Mojmir Svoboda wrote:
When the logging system initializes, it reads config info (from a file or registry data) to enable/disable particular string tags. I usually
could you show me perhaps some of your config files? just for inspiration...
In a product using a separate config file specifically for logging, the file contents might look like this: ---8<--- error warn assets config --->8--- The code splits on whitespace, adding the string tags to a map. Again, map lookup occurs only the /first/ time control reaches a particular log message. When I'm reading a more generic config file (or registry subtree), the above can go into a string-valued config variable instead. For a developer debugging session, it might look like this: ---8<--- * events memory (long list of other tags I explicitly want to suppress) --->8--- where the tag '*' means: enable all log messages except those explicitly named. This sets a flag inverting the sense of map lookup success/failure.
how do you force reload of config? on unix i'd use a signal, but to be honest, have no big clue about windows...
Heh -- since I've mostly used this with consumer products, I haven't had to confront that. You exit the program, edit the config file and start it up again. Of course that's inadequate for a general-purpose logging library. I think I'd investigate the Windows functionality that allows a program to register interest in a change to a particular file.
i planned to add another run-time filtering facility; for example you could define your facilities like that:
namespace flog {
unsigned const bit_all = bin<11111111>::value; unsigned const bit_trace = bin<00000001>::value; unsigned const bit_hedumps = bin<00000010>::value; unsigned const bit_dataflow = bin<00000100>::value; ... unsigned const bit_synchro = bin<10000000>::value;
/**@class default_filters * @brief sequence of default filters used for text formatting */ typedef runtime_flt<context, and_op > rt_context; typedef runtime_flt<level, std::less_equal> rt_level; typedef mpl::vector<rt_context ,I, rt_level ,I, time ,I, file,chr<':'>,line, I, msg>::type default_filters;
} // namespace flog
.. and then somthing like:
LOG(l, bit_debug, 7, "will enable trace..."); runtime_set(l, identity<rt_context>(), bit_hexdump | bit_trace); { ENTRY(l, 7, "void foo(T) [with T = long int]"); }
i'd really like to know your opinion on that.
I once worked on a very large product whose error codes were #defined in a central header file, e.g. #define ERROR_FILENOTFOUND 1001 ...many, many others... Every .cpp file in the system #included that file. Any change to that file forced a lengthy full rebuild on every developer. What happened was that developers quietly rebelled, hard-coding new decimal values into error-message-emitting function calls. Naturally, since those decimal values were no longer centrally registered, duplicates cropped up in different subsystems. Moreover, a typo in a hard-coded number (e.g. transposing two digits) could and did get released to the world before anyone noticed. In C++ source code, having a single, central source of truth for such things can actually be counterproductive. I much prefer a decentralized approach. An early incarnation of the log mechanism I've been describing used single-character tags: easy to code, more recognizable than decimal constants, cheap to store and compare. Scales /very/ poorly. I've come to prefer string tags, and have used that approach in several different products now. While theoretically possible, collisions are rare. Typos are readily spotted.
Nat Goodspeed wrote:
how do you force reload of config? on unix i'd use a signal, but to be honest, have no big clue about windows...
Heh -- since I've mostly used this with consumer products, I haven't had to confront that. You exit the program, edit the config file and start it up again.
I should add that my implementation depended on the fact that the set of enabled tags was constant for each run. I was storing the result of tag lookup in a static bool local to each log-message statement. For dynamic log configuration, I think I'd either have to capture a pointer to a heap bool, whose value might change, or introduce some sort of model/view paradigm to ensure that individual log-message statements would respond to a change intended to affect them.
On Tue, Jun 10, 2008 at 07:17:00AM +0300, John Torjo wrote:
At this time, my library already does this - you can certainly specify whether you want thread-safety or not, and the lib is certainly efficient.
Great. I am looking forward to the next release, but already the current version addresses my needs. Matthias -- Matthias Vallentin vallentin@icsi.berkeley.edu pgp/gpg: 0x37F34C16
Matthias Vallentin wrote:
On Tue, Jun 10, 2008 at 07:17:00AM +0300, John Torjo wrote:
At this time, my library already does this - you can certainly specify whether you want thread-safety or not, and the lib is certainly efficient.
Great. I am looking forward to the next release, but already the current version addresses my needs.
Good to know, thanks ;) Best, John
Matthias
-- http://John.Torjo.com -- C++ expert http://blog.torjo.com ... call me only if you want things done right
participants (9)
-
Hansi -
John Torjo -
Josef Grahn -
Maarten Nieber -
Matthias Vallentin -
Mojmir Svoboda -
Nat Goodspeed -
Patrick Loney -
Zeljko Vrba