[log][phoenix] Questions on porting to phoenix
Hi, [My apologies for the long post] I'm trying to port filters and formatters of Boost.Log to Boost.Phoenix and I'm not sure I understand what's the best way to go. Hope someone can help me out. A short introduction of the terms and what is currently available. * An attribute value is a type-erased value, something similar to boost::any but with more advanced interface to allow visitation and extraction of the stored value. * A log record is an object which contains a set of named attribute values. For simplicity, let's say: class attribute_value; typedef map< string, attribute_value > attribute_values_view; class record { public: attribute_values_view const& attribute_values() const; string const& message() const; }; * A filter is a function that accepts attribute values and returns a bool: typedef function< bool (attribute_values_view const&) > filter; The filter is assumed to pass the log record if it returns true. * A formatter is a function that accepts a log record and a stream and outputs strings or attribute values to the stream: typedef function< void (ostream&, record const&) > formatter; * I already have the code that extracts attribute values from log records. What I'm trying to achieve is something along these lines (this is not a finalized interface yet): enum my_severity { info, warning, error }; BOOST_LOG_REGISTER_ATTRIBUTE("Severity", severity, my_severity) filter flt = severity >= warning; formatter fmt = stream << "[" << severity << "] " << message; Here, "stream" and "message" are keywords provided by Boost.Log, the former is a placeholder for the stream argument of the formatter function, and the latter returns the result of the record::message() method. Now, the BOOST_LOG_REGISTER_ATTRIBUTE macro is what I'm struggling on. It should create a new keyword (severity) which should: 1. Be a POD object to be statically initialized. 2. Contain the information on the type (my_severity) and name ("Severity") of the attribute to be able to extract it from the log record. 3. Be a Phoenix object to initiate lazy expression generation.
From what I gathered from the docs and examples, the generated keyword should be a terminal, in terms of Proto. Am I correct? In order to produce the value, during the evaluation the keyword should receive either a log record or a attribute_values_view and I can't quite understand how do I do that. Basically, I don't quite understand how do I declare and implement a terminal so that it works with Phoenix. This question also applies to the "stream" and "message" keywords but I suspect the solution will be similar.
Maybe there is an example of doing this in the docs or elsewhere. I'll be glad if someone provides a link or example of doing this. Thanks in advance.
Hi Andrey, 2012/1/23 Andrey Semashev <andrey.semashev@gmail.com>
Hi,
[My apologies for the long post]
I'm trying to port filters and formatters of Boost.Log to Boost.Phoenix
That's exciting :-) [...] What I'm trying to achieve is something along these lines (this is not a
finalized interface yet):
enum my_severity { info, warning, error };
BOOST_LOG_REGISTER_ATTRIBUTE("Severity", severity, my_severity)
filter flt = severity >= warning; formatter fmt = stream << "[" << severity << "] " << message;
Here, "stream" and "message" are keywords provided by Boost.Log, the former is a placeholder for the stream argument of the formatter function, and the latter returns the result of the record::message() method.
Now, the BOOST_LOG_REGISTER_ATTRIBUTE macro is what I'm struggling on. It should create a new keyword (severity) which should:
1. Be a POD object to be statically initialized. 2. Contain the information on the type (my_severity) and name ("Severity") of the attribute to be able to extract it from the log record. 3. Be a Phoenix object to initiate lazy expression generation.
I wrote the sample code that conforms these requirements, full code attached.
From what I gathered from the docs and examples, the generated keyword should be a terminal, in terms of Proto. Am I correct?
I think it's a phoenix actor which is also a proto terminal. In order to produce the value,
during the evaluation the keyword should receive either a log record or a attribute_values_view and I can't quite understand how do I do that. Basically, I don't quite understand how do I declare and implement a terminal so that it works with Phoenix. This question also applies to the "stream" and "message" keywords but I suspect the solution will be similar.
Maybe there is an example of doing this in the docs or elsewhere. I'll be glad if someone provides a link or example of doing this. Thanks in advance.
HTH
On Mon, Jan 23, 2012 at 2:37 PM, TONGARI <tongari95@gmail.com> wrote:
I wrote the sample code that conforms these requirements, full code attached.
Thanks a lot, this will greatly help me to get started. So, as I understand, the env argument to the actor contains arguments provided for evaluation. In case of filtering expression, this is fine as there is only one argument - the set of attributes. But in case of formatter there are two arguments - the stream and the log record. Does Phoenix pass both arguments to the actor in this case?
On Mon, Jan 23, 2012 at 1:57 PM, Andrey Semashev <andrey.semashev@gmail.com> wrote:
On Mon, Jan 23, 2012 at 2:37 PM, TONGARI <tongari95@gmail.com> wrote:
I wrote the sample code that conforms these requirements, full code attached.
Thanks a lot, this will greatly help me to get started.
So, as I understand, the env argument to the actor contains arguments provided for evaluation. In case of filtering expression, this is fine as there is only one argument - the set of attributes. But in case of formatter there are two arguments - the stream and the log record. Does Phoenix pass both arguments to the actor in this case?
Yes, the environment is a fusion tuple. It is layouted as follows: +------------------------------+---------+ | Arguments: | Actions | | +------+------+ ... +------+ | | | | Arg0 | Arg1 | ... | ArgN | | | | +------+------+ ... +------+ | | +------------------------------+---------+ where Arguments can be accessed with env.args() or phoenix::context(env) Actions are the proto external transforms which determine how the expression will be evaluted. I have to admit the docs are very unclear about that specifc part ... needs some loving.
On Mon, Jan 23, 2012 at 5:07 PM, Thomas Heller <thom.heller@googlemail.com> wrote:
On Mon, Jan 23, 2012 at 1:57 PM, Andrey Semashev <andrey.semashev@gmail.com> wrote:
On Mon, Jan 23, 2012 at 2:37 PM, TONGARI <tongari95@gmail.com> wrote:
I wrote the sample code that conforms these requirements, full code attached.
Thanks a lot, this will greatly help me to get started.
So, as I understand, the env argument to the actor contains arguments provided for evaluation. In case of filtering expression, this is fine as there is only one argument - the set of attributes. But in case of formatter there are two arguments - the stream and the log record. Does Phoenix pass both arguments to the actor in this case?
Yes, the environment is a fusion tuple.
Ok, so it seems the attribute actor will have to be an unary function and always expect either a log record or a set of attribute values as the first argument. It's fine, as long as the argument can be rebound with phoenix::bind. A few more questions, if I may. 1. I would like to cache some data in the actor at the point of AST construction. For instance, I'd like to cache the attribute name (in the actual code it's not really a string but rather an adapter constructible from a string; I'd like to avoid its construction at the expression evaluation time) and, possibly, a default value for the attribute. Is this possible? The attribute keyword should still be a POD (or at least, be safely constructible at the namespace scope), so the cached data can only be provided only by a function call. 2. I would like the keyword have additional methods, in particular, for default attribute value provision. Extending my initial syntax, this would be: BOOST_LOG_REGISTER_ATTRIBUTE("Severity", severity, my_severity) attribute_values_view attrs; severity(attrs); // returns optional<my_severity> severity.or_default(info)(attrs); // returns my_severity, which is info by default As I understand, the only way I can do that, except for reimplementing phoenix::actor, is to derive my actor from phoenix::actor. This makes the attribute keyword non-POD in C++03 but is it still safe to be constructed in the namespace scope, provided that the keyword does not have its own data members? Also, what should be the template argument for the phoenix::actor specialization? Here's the code example to illustrate what I'm talking about: template< typename DescriptorT > struct attribute_keyword : public phoenix::actor< what_should_be_here? > { // extracts the attribute value template< typename ContextT > DescriptorT::value_type operator() (ContextT const& ctx) const; // adds a default value attribute_keyword_with_default< DescriptorT > or_default(DescriptorT::value_type) const; }; // assume there are specializations of custom_terminal and // is_custom_terminal for attribute_keyword // This is generated by BOOST_LOG_REGISTER_ATTRIBUTE namespace tag { struct severity { typedef my_severity value_type; typedef attribute_name name_type; // returns an equivalent for "Severity" // I want to cache its result in the AST static name_type get_name(); }; } const attribute_keyword< tag::severity > severity; // is this safe?
I have to admit the docs are very unclear about that specifc part ... needs some loving.
Right, the docs are lacking a good tutorial for library extenders. Its internals are rather complicated to gather this information from.
2012/1/24 Andrey Semashev <andrey.semashev@gmail.com> [...]
A few more questions, if I may.
1. I would like to cache some data in the actor at the point of AST construction. For instance, I'd like to cache the attribute name (in the actual code it's not really a string but rather an adapter constructible from a string; I'd like to avoid its construction at the expression evaluation time) and, possibly, a default value for the attribute. Is this possible? The attribute keyword should still be a POD (or at least, be safely constructible at the namespace scope), so the cached data can only be provided only by a function call.
2. I would like the keyword have additional methods, in particular, for default attribute value provision. Extending my initial syntax, this would be:
BOOST_LOG_REGISTER_ATTRIBUTE("Severity", severity, my_severity)
attribute_values_view attrs;
severity(attrs); // returns optional<my_severity> severity.or_default(info)(attrs); // returns my_severity, which is info by default
As I understand, the only way I can do that, except for reimplementing phoenix::actor, is to derive my actor from phoenix::actor. This makes the attribute keyword non-POD in C++03 but is it still safe to be constructed in the namespace scope, provided that the keyword does not have its own data members?
I'm not sure as well, that's what I always want to ask Eric: http://www.boost.org/doc/libs/1_48_0/doc/html/proto/appendices.html#boost_pr... Anyway, I played a trick while not knowing if it's available. The idea is: struct Pod {...you could have some data...}; struct Derived : Pod {}; // non-POD in C++03 Pod p_base = {...some data...}; // statically initialized Derived& p = static_cast<Derived&>(p_base); // statically initialized, not? Also, what should be the template argument
for the phoenix::actor specialization? Here's the code example to illustrate what I'm talking about:
[...] Again, I wrote the sample code (a rewrite from my last sample). I store the attribute name, this way, you don't need a new type for each keyword (which could be of the same value_type). Oh, BTW, I hope I can have severity(attrs) return optional<my_severity const&> instead of optional<my_severity> in case that my_severity is heavy (not this case, I know), but then the comparison ops are lost :( HTH
On 1/24/2012 1:54 AM, TONGARI wrote:
2012/1/24 Andrey Semashev <andrey.semashev@gmail.com>
2. I would like the keyword have additional methods, in particular, for default attribute value provision. Extending my initial syntax, this would be:
BOOST_LOG_REGISTER_ATTRIBUTE("Severity", severity, my_severity)
attribute_values_view attrs;
severity(attrs); // returns optional<my_severity> severity.or_default(info)(attrs); // returns my_severity, which is info by default
As I understand, the only way I can do that, except for reimplementing phoenix::actor, is to derive my actor from phoenix::actor. This makes the attribute keyword non-POD in C++03 but is it still safe to be constructed in the namespace scope, provided that the keyword does not have its own data members?
I'm not sure as well, that's what I always want to ask Eric: http://www.boost.org/doc/libs/1_48_0/doc/html/proto/appendices.html#boost_pr...
Proto doesn't force you to use inheritance to extend and expression with additional member (see BOOST_PROTO_EXTENDS). But Phoenix may, I don't know. Thomas?
Anyway, I played a trick while not knowing if it's available. The idea is:
struct Pod {...you could have some data...}; struct Derived : Pod {}; // non-POD in C++03 Pod p_base = {...some data...}; // statically initialized Derived& p = static_cast<Derived&>(p_base); // statically initialized, not?
Aside from the fact that this is non-standard, not all compilers statically initialize references, though they should. -- Eric Niebler BoostPro Computing http://www.boostpro.com
On 1/25/2012 12:30 AM, Eric Niebler wrote:
Proto doesn't force you to use inheritance to extend and expression with additional member (see BOOST_PROTO_EXTENDS). But Phoenix may, I don't know. Thomas?
AFAIU it's the same basic design as Phoenix-2. There is still an actor base class. But there should be a way to use bare proto terminals and have them taken in using as_actor. Regards, -- Joel de Guzman http://www.boostpro.com http://boost-spirit.com
On Wednesday, January 25, 2012 08:42:13 Joel de Guzman wrote:
On 1/25/2012 12:30 AM, Eric Niebler wrote:
Proto doesn't force you to use inheritance to extend and expression with additional member (see BOOST_PROTO_EXTENDS). But Phoenix may, I don't know. Thomas?
AFAIU it's the same basic design as Phoenix-2. There is still an actor base class. But there should be a way to use bare proto terminals and have them taken in using as_actor.
I think I managed to do this as Eric suggested: template< typename DescriptorT, typename ExprT = phoenix::actor< attribute_terminal< DescriptorT > > > struct attribute_keyword { BOOST_PROTO_EXTENDS(ExprT, attribute_keyword, phoenix::phoenix_domain) }; This way the keyword infects the expression as expected. What about as_actor? How do I use it?
On Tuesday, January 24, 2012 17:54:15 TONGARI wrote:
Again, I wrote the sample code (a rewrite from my last sample). I store the attribute name, this way, you don't need a new type for each keyword (which could be of the same value_type).
Thank you, you're very helpful. I'll take a look and get back.
Oh, BTW, I hope I can have severity(attrs) return optional<my_severity const&> instead of optional<my_severity> in case that my_severity is heavy (not this case, I know), but then the comparison ops are lost :(
Unfortunately, this is not possible ATM. As I remember, this limitation was because an attribute value object might not actually store the value but rather generate it on visitation (i.e. the value is passed to the visitor by reference and then destroyed). I can't remember the exact case where this appeared, I'll have to take a look at the code.
On 1/24/2012 2:41 PM, Andrey Semashev wrote:
As I understand, the only way I can do that, except for reimplementing phoenix::actor, is to derive my actor from phoenix::actor. This makes the attribute keyword non-POD in C++03 but is it still safe to be constructed in the namespace scope, provided that the keyword does not have its own data members? Also, what should be the template argument for the phoenix::actor specialization? Here's the code example to illustrate what I'm talking about:
template< typename DescriptorT > struct attribute_keyword : public phoenix::actor< what_should_be_here? > { // extracts the attribute value template< typename ContextT > DescriptorT::value_type operator() (ContextT const& ctx) const;
// adds a default value attribute_keyword_with_default< DescriptorT > or_default(DescriptorT::value_type) const; };
// assume there are specializations of custom_terminal and // is_custom_terminal for attribute_keyword
// This is generated by BOOST_LOG_REGISTER_ATTRIBUTE namespace tag { struct severity { typedef my_severity value_type; typedef attribute_name name_type;
// returns an equivalent for "Severity" // I want to cache its result in the AST static name_type get_name(); }; } const attribute_keyword< tag::severity > severity; // is this safe?
This has been pretty much the practice in phoenix (and even in spirit) for placeholders and such. We don't recommend this now for several reasons. One big reason is that the proliferation of such namespace scope proto objects (there's a lot in spirit), even if PODs, slow down compile time considerably (esp. on g++ as reported by Joel Falcou et.al) (note: apparently, only proto objects are affected by this). Now, what I advocate is exposing the typedef: attribute_keyword< tag::severity > severity_type; and letting the user create a local (function scope) object himself: boost::log::severity_type severity; The typing involved is the same. With namespace scope objects, the "best-practice" is for you to put these objects in a spacial namespace and hoist it into the client namespace using using declarations anyway (e.g.): using boost::log::severity; The compile time problem is very significant that in Spirit and indeed in Phoenix-3, we have special PP defines to disable the predefined objects (e.g.): #define BOOST_SPIRIT_NO_PREDEFINED_TERMINALS Regards, -- Joel de Guzman http://www.boostpro.com http://boost-spirit.com
On Wednesday, January 25, 2012 08:20:02 Joel de Guzman wrote:
Now, what I advocate is exposing the typedef:
attribute_keyword< tag::severity > severity_type;
and letting the user create a local (function scope) object himself:
boost::log::severity_type severity;
The typing involved is the same. With namespace scope objects, the "best-practice" is for you to put these objects in a spacial namespace and hoist it into the client namespace using using declarations anyway (e.g.):
using boost::log::severity;
The compile time problem is very significant that in Spirit and indeed in Phoenix-3, we have special PP defines to disable the predefined objects (e.g.):
#define BOOST_SPIRIT_NO_PREDEFINED_TERMINALS
Interesting, thanks for the info. But these keywords are to be user-defined and the whole point behind them is to reduce typing and simplify Boost.Log usage. For instance, I also plan to support this syntax for extracting an attribute value from a log record: optional< my_severity > sev = rec[severity]; Having to declare a local keyword variable for this seem inappropriate. But I can make the keyword creation optional by providing several macros for keyword registration - one that only defines the keyword type and another one that also creates the keyword. Also, I can put the declared keyword in a namespace and let the user decide whether he wants to import it by using declaration or not.
participants (5)
-
Andrey Semashev -
Eric Niebler -
Joel de Guzman -
Thomas Heller -
TONGARI