Hi Gareth and others, ----- Original Message ----- From: "Gareth Buxton" <gareth.buxton@googlemail.com> To: <boost@lists.boost.org> Sent: Monday, April 09, 2007 9:13 PM Subject: Re: [boost] A meta-proposal for logging
Hi Andrey,
You see, in your example the filter would get "maskable& line", not "line_number& line" as the logging library doesn't know about "line_number" class. This means that you will have to make some type dispatch to get the real attribute type, and I'm afraid this will be a performance killer.
What constitutes death? i.e. how do you know what number of additional CPU cycles would change the mind of a potential adopter. There is no real boundary. It is way too dependent on the circumstance.
It could be but I don't think it necessarily has to. My solution would be to tightly couple the filter with the attributes that are to be filtered on the basis that if the user is adding the attributes then the use knows how best to filter based on them.
<snip>
I tend to think that some amount of coupling is inevitable, for the most part I would imagine attributes would be well known, frequently used types like time (time_t), line_number (int) etc. and would be provided as 'standard' in the library.
I would have liked to respond to the individual points raised here but there are a lot of them. If there is something useful I can add it is most likely a brief summary of a logging system recently completed. I balanced many of the issues raised here. * Logging is a stream of mixed-type records or discriminated variants. * In application code, logging looks like; log_channel << "Reached here"; log_channel << file_IO_problem; log_channel << server_snapshot(); log_channel << stack_trace(); * The stream of objects is asynchronously queued, sent over a network connection, collated, time stamped and stored using a proprietary format and file management scheme. * A dedicated viewer allows entry of arbitrary filtering expressions that typically results in a window-per record type, i.e. a fixed set of attributes. * Users can navigate inside the filtered view by entering further search criteria or by clicking on an auto-generated time breakdown of the view. * The record types are "user defined". Registration of a type includes enough information for the viewer to display full attribute information. * The logging applications (i.e. the source of log records) communicate with the server using a formalized protocol. An obvious message (or signal) within this protocol is LOG (i.e. the connecting client is transferring a log record, an instance of a discriminated variant). * Other less obvious protocol messages involve the distribution of the registered types, i.e. the logging application publishes its type information to the logging server. * The logging server accumulates and merges all the type information from all the different sources. * The logging server persists its own "super-symbol-table" to reduce traffic and also to enable the display of type information when no logging applications are connected. This might be termed a "full implementation" of logging. Some high-end requirements have certainly dragged it a long distance from where it started. Some possibly useful lessons picked up along the way; * The software machinery that provides for creation, transport, queueing, storage, retrieval and viewing of logs is far more important than a particular set of attributes. * There is no ultimate set of attributes. The set of attributes on this project changed after almost every discussion and remains in the state "to be determined". * Many of the initial requirements were eventually shown to be silly. Even now it remains very difficult to convince interested parties of this, e.g. a live window of received log records is not useful. * There are huge benefits in creating a logging system that transports and stores generic blobs (mixed-type log records). All the significant related software is "logging application" independent and inherently "forward compatible". * The cost in software complexity and dependence on formalized communcations and storage mechanisms is non-trivial. * Given another opportunity and certainly in the case of a Boost.Log implementation I would probably swing away from the "published type info" approach and go for straight text. This is mainly due to the amenable nature of text (much easier to get a google search implemented) and general software simplicity. The costs include space resources (RAM consumed in log queues and disk space) and some loss of viewing+filtering performance. Cheers.