Re: [boost] [Review] Boost.Logging: formal review

-----Original Message----- From: boost-bounces@lists.boost.org [mailto:boost-bounces@lists.boost.org] On Behalf Of Caleb Epstein Sent: den 19 december 2005 16:41 To: boost@lists.boost.org; Gennadiy Rozental; John Torjo Subject: Re: [boost] [Review] Boost.Logging: formal review
I personally prefer the all-enclosed macro style for this reason as well as others. But I think one of Gennadiy's points is that the primary interface to the Log should not be defined in terms of macros. A standard set of macros can be provided by the Boost.Log implementation for the sake of convenience and casual use. But there should be a well documented public interface that users with more complex requirements can use to write their own logging macros, in whichever style they prefer.
Yes, this is a must. The "basic" interface should be quite minimalistic and simple to use. The interface should not have any domain specific nor "too much preferred logging styles". For instance, I don't think that Gennadiy's entry keywords belong directly in this interface. There would always be a domain or user that wants specific details to be added, details that might not be a concern at all in my application. However, an entry keyword should still be very simple to "implement/use" on top of the basic interface.
I don't think it makes sense to require a user to have separate loggers to handle e.g. DEBUG, WARNING, and ERROR levels. I also don't think it makes sense to require separate logs for different logging keywords (I think thats the term Gennadiy used). The user may use separate loggers for each of these separate logical channels, but it should not be a requirement.
I'm not saying that this is required. I'm saying that it should be possible. I guess what I am saying is (like my paragraph above) style should be left to programmers and not be enforced. Especially, a logger must be able to have zero, one, or several, sinks (appenders) attached to it, and I think that that is something that you have addressed later in your post.
IMO it is very important for an administrator/programmer of a system to
be able to enable/disable logs, know what loggers and appenders exists and be able to configure said entities.
But if the programmer has to provide a separate logger for each logging level and conceptual channel of log information, each of which is configured independently, isn't this management problem far worse?
I suppose it could be a management nightmare and if this is the case, one would need to have fewer loggers (or one if you like), in which case a logger is not tied to a specific log level (e.g.). And in this case, the programmer would use whater log functions deemed necessary to make the job simple(r). However, I'm not sure such functions should be part of the logger interface, but could be functions/macros that depend on the loggers and not vice versa in order to minimize the logger's interface.
If filter is only done on a per log statement, how do you even know as a
administrator of a system what log filters exists? Which filter criteria should be enabled and which filter criteria should be disabled?
Can you expound a bit on what you mean by this?
Well, with a development team of 20+ people I might want to tightly control what is being logged and that all "important" log messages are sent to a specific sink. If entry keywords are used, for instance, I'm quite sure that the number of keywords in use probably would explode. Which of these are important and which are not so important? Which are warnings, which messages should be filtered and so on. Perhaps I want to enable all log messages with a the specific entry keyword "my_special_condition", but what happens when a programer misspelled this keyword. This could also turn into a management nightmare IMO (unless I'm totally misunderstanding the concept). If a systems architect would like to send specific log message to specific sinks, wouldn't this be a management problem as well? I would still have to setup and configure loggers/filter/sink combinations. I'm not sure there is a "better" solution to this general problem. It's a difficult problem and should be treated as such. IMO, there must be several ways to solve the issue, chosen by programmers and teams that suits their specific "design" choice. This boils down to, IMO, to simplistic loggers and sinks, and a more "complex" extensible layer on top of this, perhaps provided by the library for common situations..
I see the central concepts as:
Entry (or Message): the set of information that makes up a single message to log. Might include a "level" (e.g. debug, info, warning, error), a "category" (e.g. arguments, program_flow, etc), a "keyword" (e.g. "Network", "UI", "Disk Cache", etc) and of course a text string.
Could be more or it could be less in specific situations. But other than that I agree with you.
Filter: determines whether or not an Entry should be logged. Probably implemented as a relatively simple functor. Sink: ultimate destination for Entries. Implementations for wirting to files, system log facilities, etc can be provided. Log: provides logging methods. Implements the Filter concept and contains zero or more Sinks which are the targets of Entries for which the Filter returns true.
I aggree with you on a conceptual level. However, the Filter concept shouldn't be part of, IMO, the logger. Or if it is, at least, there should be an alternative interface in the logger, or possibly, the filtering concept should be very simple if I don't want entry keywords and such. Fundementally, I think that writing log statements and filtering log statements are two different and orthogonal concepts. Filtering is an efficiency decision at the conceptual level. With that I mean if we had infinite computing power (which we don't) filtering wouldn't be necessary at all when sending log entries to sinks. Filtering is however something the target audience does when looking for, or at, log statements. That is, with infinite computing power, the filtering mechanism could be done at the time when administrators/programmers are looking/searching logs. But since we don't have infinite computing power, log entries need to be filter before sending them to sinks (I suppose one could argue that there are other reasons for filtering before sending entries to sinks; but I don't see a difference). So the interface for enabling/disabling log entries should be a fundemental concept to a logging library and obviously as well designed as the actual log entry interface. Especially, wether to send a log entry to a sink should incur minimal overhead, or else risk efficiency even if the log statement are not sent to the sinks. This is especially true if programers use many log statements. IMO at the most fundemental layer, a logger is either enabled or disabled. If it is enabled, log entries are sent to its attached sinks. Filtering should be implemented on top of loggers. The filtering concept determines wether to send the log entry to the logger or not.
I've been sketching up a very simple implementation of some of these ideas in my spare time. Is there interest in fleshing these ideas out more fully on or off-list?
Cool! Yes, very interested (I have myself an implementation... who doesn't ;-)) /Richard The content of this e-mail is intended only for the confidential use of the person(s) to whom it is addressed. If the reader of this message is not such a person, you are hereby notified that you have received this communication in error and that reading it, copying it, or in any way disseminating its content to any other person, is strictly prohibited. If you have received this message in error, please notify the author by replying to the e-mail immediately. NeoNet AB and its subsidiaries (NeoNet Securities AB, NeoNet Securities Inc. and NeoNet Technology AB) are unable to exercise control over the content of information contained in transmissions made via the Internet and hereby excludes any warranty as to the quality or accuracy of any information contained in this message and any liability of any kind for the information contained in it, or for its transmission, reception, storage or use in any way whatsoever.

I see the central concepts as:
Entry (or Message): the set of information that makes up a single message to log. Might include a "level" (e.g. debug, info, warning, error), a "category" (e.g. arguments, program_flow, etc), a "keyword" (e.g. "Network", "UI", "Disk Cache", etc) and of course a text string.
Could be more or it could be less in specific situations. But other than that I agree with you.
Filter: determines whether or not an Entry should be logged. Probably implemented as a relatively simple functor. Sink: ultimate destination for Entries. Implementations for wirting to files, system log facilities, etc can be provided. Log: provides logging methods. Implements the Filter concept and contains zero or more Sinks which are the targets of Entries for which the Filter returns true.
I aggree with you on a conceptual level. However, the Filter concept shouldn't be part of, IMO, the logger. Or if it is, at least, there should be an alternative interface in the logger, or possibly, the filtering concept should be very simple if I don't want entry keywords and such. Fundementally, I think that writing log statements and filtering log statements are two different and orthogonal concepts. Filtering is an efficiency decision at the conceptual level. With that I mean if we had infinite computing power (which we don't) filtering wouldn't be necessary at all when sending log entries to sinks. Filtering is however something the target audience does when looking for, or at, log statements. That is, with infinite computing power, the filtering mechanism could be done at the time when administrators/programmers are looking/searching logs.
But since we don't have infinite computing power, log entries need to be filter before sending them to sinks (I suppose one could argue that there are other reasons for filtering before sending entries to sinks; but I don't see a difference). So the interface for enabling/disabling log entries should be a fundemental concept to a logging library and obviously as well designed as the actual log entry interface.
Especially, wether to send a log entry to a sink should incur minimal overhead, or else risk efficiency even if the log statement are not sent to the sinks. This is especially true if programers use many log statements.
IMO at the most fundemental layer, a logger is either enabled or disabled. If it is enabled, log entries are sent to its attached sinks. Filtering should be implemented on top of loggers. The filtering concept determines wether to send the log entry to the logger or not.
Filterring is indeed is in theory is orthogonal to formatting. With two simple but very important exceptions: 1. The data that used for filterring is as frequently needed for formatting and we do not want to collect data twice. I think that Filter concept should be extended (or another concept needs to be introduced: Non-Filterring Entry component). 2. Log need to know entry data components by name to be able to configure formatting properly While designing one needs to keep in mind following points: 1. There are entry components that are used for filterring. These should be collected before the filterring stage begins. 2. There are entry components that are not used for filterring. These should be collected before the formatting stage begins (pre/post filterring is not mandated). Any of the components we talked previosly - level, file, line keyword, thread id - could, depending on users, belong to either group. 3. We do not want to collect data for filterring and formatting twice. 4. Log should be able to refer to the entry components by name for configuration purposes. I want to be able to configure log/sink with config string: "prefix=thread_id time file(line):" 5. Sone entry components need external source for initialization (file, line, level etc), some doesn't (thread id, time). Gennadiy

Hi All, I think the formatting should be user specified one. After an Entry goes through filter and is being ready to be written to Sink I think that's when we need to perform formatting. And for that to happen the formatter could be functor which a user can specify for each sink. Because each could have a different way of formatting the message. This provides a high level of flexibility to the user to choose sink and formatter. Question: Should the logger concept be able support multi-threaded environment logging. -- Nitin Motgi NITIN . DOT. MOTGI .AT. GMAIL .DOT. COM

On 12/20/05, nitin motgi <nitin.motgi@gmail.com> wrote:
Question: Should the logger concept be able support multi-threaded environment logging.
Definitely. What might be a better question however is should the logger be thread aware by default? -- Caleb Epstein caleb dot epstein at gmail dot com
participants (4)
-
Caleb Epstein
-
Gennadiy Rozental
-
nitin motgi
-
Richard Glanmark