John Torjo <john.lists <at> torjo.com> writes:
Dear boosters,
Worked some more on the logging library
Hi John, I really like your idea of simplifying the actual logging by not directly modeling the hierarchy. The if/else macro to only conditionally evaluate the arguments is also an essential feature that your design includes. However, I think one significant feature is missing, and some aspects of the design and implementation don't seem to suit at least one use (mine of course :-). However, logging seems to be one of those things that everyone at least thinks they have special requirements for, and maybe I'm missing how to apply the library effectively. I think a *small* library is desirable, and any usage dependant issues should be dealt with by making the library extensible/configurable, not larger. The specific features I'm concerned about... Support for dynamic log creation: Some form of dynamic log creation would be very useful. Consider a networked system of some kind in which the application instantiates an object with a runtime determined name/identifier (such as the address/name of the peer/client/server/node/service/entity). So to turn on detailed logging of information from just one such entity, named "bob" one would (via an appropriate runtime UI) enable_logs("app.trace.client.bob"). The object that represents a connection with bob needs to hold some scoped_log object. The actual writing to the log would need to be different for a dynamic vs static log - a macro taking the scoped_log rather than the log_name. Possible bug in handling of early logging: Isn't the effectiveness of the facility to buffer data written prior to defining the log dependent on the order of instantiation of the static objects that log vs the instantiation of the logid's themselves? If any logging is attempted after the logid name is registered but before any log_funcs have been added the messages will be lost. Or am I missing something here? Log enable/disable/add/remove performance/scalability: The separation of logging and the implied hierarchy has made for a relatively expensive update process when the logging settings are changed. As you point out in the docs, this shouldn't be a big issue, as they don't change often. However, I am somewhat disturbed by what looks like an O(n^2) cost for log enable/disable. Maybe this is just an implementation detail that can be optimised relatively easily (couldn't the raw enabled array be implemented as a map keyed by prefix?). One could also imagine building a more elaborate, data structure to represent the hierarchy if this level of optimisation was really needed (I'm not suggesting that you should do this). Multi-threading performance: The caching of the loglib_info on a per thread basis seems to have the potential to produce some large latencies in the operation of a heavily-multi- threaded application whenever logging settings are changed. The loglib_info is certainly not a lightweight object and is copied between threads (while holding a lock on the master/global instance). In addition to the copying, just the storage used per thread due to the duplication of data may be significant for an application with a lot of lightweight threads. I'm also not convinced that your lock-free counter is strictly portable, but in practice I don't know of a system it wouldn't work well enough on. Regardless, I think it is solving the wrong problem (at least for systems with any form of real-time constraints) in that the whole design seems to offer low average latency, but with a worst-case latency that depends on and scales poorly with the size of the system (measured by number of logs and number of threads). Finer-grained locking would seem preferable. A version that explicitly forgoes thread safety would also seem useful - and not just for single threaded apps. It seems like a lot of the implementation is tied up in the thread support area - it would be nice to separate the threading issues more cleanly. Efficient long/complex log entries:
From the docs: // to avoid this: // (which could be very time-consuming - acquiring the log one million times) for (int idx = 0; idx < 1000000; ++idx) BOOST_LOG(app) << "message at idx " << idx << std::endl;
Not only time consuming - potentially, only some lines of output would be generated (if log settings changed) and other log output could be interleaved between lines of this message. // you will want to do this: // (very efficient) if (BOOST_IS_LOG_ENABLED(app)) { std::ostringstream out; for (int idx = 0; idx < 1000000; ++idx) out << "message at idx " << idx << std::endl; BOOST_LOG(app) << out.str(); } Much better - but then why can't I do this: if (MAKE_LOG(log, app)) { for (int idx = 0; idx < 1000000; ++idx) log << "message at idx " << idx << std::endl; } Which should be more efficient again? Regards Darryl.