On Mar 1, 2010, at 1:03 PM, Matthias Vallentin wrote:
On Mon, Mar 01, 2010 at 12:03:57PM -0500, Daniel Larimer wrote:
If you change the abstraction to nothing but a stream of data packets of "some type" that flows from "some source" through various "filters" to "one or more" destinations, then a "log file" becomes little more than a "buffer" in the stream.
Thus, if you have some "real time monitoring tools" that can be hooked up to debug then these tools should be agnostic to whether or not the channel they receive their data on is from the original source or from the other side of a "log file filter".
It looks like the replay tool performs some sort of input parsing on a given stream. What I am trying to understand is whether you are asking to couple those two processes. That is, do you advocate that the definition of the output format should automatically generate a parser (e.g., via Spirit) for the output?
Lets assume you have a data source (boost::signal) and you connect it to a chain of filters that add meta data and ultimately a filter that converts the types into an archive and finally one that buffers "archives to disk". Another filter would simply take an archive and ultimately reproduce a boost::signal. You still need to have a "filter" that can convert a binary or text log entry back into the original types, but the process of fetching individual log entries and controlling how and when they are "emited" could be entirely handled. Now if the logging library loses all concept of "record boundary" when it writes to disk then there is not much you can do except create a custom log reader that determines the boundary of a specific record. But I contend that for many people they simply want to say "log this signal to this file under this name" and then open their log and say "replay signal with name to this slot" and then iterate through the log entries possibly in a multiple of real time. This opens up a ton of existing code which is already separated into "producer / consumer" or "model / view" paradigms via signals to logging and visualization of logged data without adding any new dependencies to the original code. I do not mean to put my requirements on this if my requirements are "out of scope" or to insist that such a feature is necessary out of the box, only that a boost logging library should be flexible enough to support some means of iterating over records in a log file. Another reason to support this kind of function is when logging a large amount of data at high data rates.... binary files would be an order of magnitude smaller, result in less disk access, and use less "formatting" overhead than text files. Yet it should be "trivial" to convert a binary log to a text log simply by using the source => serialization => binary file => binary to obj => text sink. If your library assumes that ostream<< is defined for every type then making an assumption on boost::serialization to support such a feature would be a relatively minor addition. If my "requirements" fall outside the scope of 'logging' then perhaps we should define what logging means. One last consideration is that often there is a desire to separate logging into a separate process that can optionally "connect" to the logging core and possibly perform some additional filtering on the other side of a socket. So my question is, how hard would it be to accomplish some of my requirements with the built in functions vs how much would I have to write from scratch to support it? I will look into it more, but perhaps someone more familiar with the package could offer an "estimate" in terms of hours required to implement some or all of the features I suggest. If they fit nicely into the framework, then I would likely contribute my own time to such an effort. If the framework starts to "get in the way" then perhaps we should consider the design. By the way, the library looks like it has great potential to manage debug print statements when layered with appropriate macros.
In general, I would love to see the library in Boost. My logging facilities are built on top of the current SVN version of the library and became an invaluable and flexible tool throughout my development.
Matthias -- Matthias Vallentin vallentin@icsi.berkeley.edu http://www.icir.org/matthias _______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost