On Aug 3, 2011, at 7:02 AM, Julien Nitard wrote:

Hi Daniel,

Some more input on my usage of boost::reflect:

- Annoying warning in reflect.hpp line 84 (multi-line comment), triggered because there's a back slash at the end of a single line comment. Removing the back slash or the line solves the problem. (I use -Wall).

I removed the line.

- Additional facility to recurse: There are two cases where I think boost::reflect could add interesting functionality when visiting a type, for other types that are themselves reflected and for arrays. I wrote a visitor that converts the endianness of a reflected struct. When I browse an object, if the member is an array then I write special logic to apply the correction on array members and on members that are themselves reflected. In that special case I print a compilation error when a member's type is not reflected. A sample code is attached. I did something similar in my CSV visitor to output member arrays as Col1 Col2 ... ColX (with an exception for char[] that is printed as a string).

A recursive visitor would require an instance of the type and would need to pass the 'this' pointer along with the member pointer and the name to the operator(). Earlier visions of the library had such an interface. I suppose it could be useful. In my primary 'test case', boost/rpc/json.hpp I also have to solve the recursive problem and I do it in a manner similar to your solution. It would visit all of your data, but you would not know where/when a nested call began unless the visitor also provided hooks. Perhaps it does not matter in your case (very interesting use case indeed!) I would like to include your code as an example, but unfortunately, it has other dependencies.

- Remove the const in operator() The above example has a good reason to modify the data while recursing, that made declare my target object has mutable which I think is awkward. There could be two constructors to visitor one taking a const object and one taking a non-const object to keep the maximum const correctness when possible.

I didn't want it to be const, however, the options are either to pass by &, const &, or value. If I remove the const then my options are & or value. If I choose & then you cannot pass a temporary visitor as a parameter and if I choose value then I must create a copy of your visitor. Ultimately I went with the const method because that is how boost::fusion visitors worked and if it is good enough for boost::fusion then it is good enough for me… right? The only other alternative is to do a const_cast in reflector<T>::visit() before calling operator(). This would bypass the errors associated with passing a reference to a temporary, but violate people's expectations of a const& param. A mutable reference makes no sense: "The mutable specifier can be applied only to names of class data members (9.2) and cannot be applied to names declared const or static, and cannot be applied to reference members." clang and maybe other compilers will complain if you create a mutable reference. I made that mistake myself. It should work if you simply remove the mutable keyword or replace the reference with a pointer. The pointer may be 'const', but what it points to will still be non-const. This still causes problems if you want to keep a counter or some other variable 'in the visitor', in that case, mutable seems to be the least-bad option. Using rvalue references could be a solution if it c++0x was an option. Unfortunately, c++0x still rules out too many users. Thanks for all the feedback, I am glad that you are finding the library so useful.

On Aug 4, 2011, at 9:23 PM, Julien Nitard wrote:

I have another use case :

After having written the byte swapper and csv output, the next logical application for reflection is a parser. The input is something of the form: { { message_type = name } | key1=value1 | key2={ ... } | key3=value3 | ... }

That looks very similar to json, put quotes around the keys and replace | with ',' and '=' with ':' and you have json. Using my own JSON parser using Spirit.Qi (soon to be converted to utree from variant) https://github.com/bytemaster/Boost.RPC/blob/master/include/boost/rpc/json/v... Using Json Spirit, using Spirit Classic https://github.com/bytemaster/Boost.RPC/blob/master/include/boost/rpc/json.h... I had an idea where I would 'auto-generate' a Spirit parser for an object that used the field names as symbols and dumped the result right into the proper field. This would then allow me to parse JSON objects into c++ structs faster than anything else out there. (no need for O(n) key lookup or temporary generic json container) The downside would be the compile times associated with having so many spirit parsers.

All this to say that boost::reflect seems to have a lot of patterns ... and I think there's work to be done in the library to factor some boilerplate code, but I am not sure how it can be done (if at all). I think at least you could document the ones you think will be used more often. I also think that the calculator example though impressive, is a bit obscure for a newcomer to your library.

The serialization pattern is one that my RPC library will use in many places: 1) JSON 2) RAW (raw bytes, no special encoding), perhaps RAW with byte swapping 3) Protocol Buffers 4) XML (several variations) 5) Conversion to Lua or Python objects in embedded scripting engines. 6) Your CSV example All of these have a very similar pattern with certain 'primitives' that require custom support and everything else is based upon recursive calls to the reflector. Finding a way to factor this pattern out would significantly help development of my RPC library.

The next thing that I'll be working on is that I need to map the names of the struct to their string counterpart as the first type of the parsing process, and here again it has to be done manually. That is I'll have to find a way to list my types in a map <string,type> while I am already calling BOOST_REFLECT that could do it at the same time (or at least I think it can, I didn't try yet). I think it is one pattern that would be a very useful addition.

I suspect that the following is not what you wanted: std::string name = boost::reflect::get_typename<type>() What I think you are asking for from the reflection library is some kind of factory that given a "type" will create a new type? There are a lot of challenges with this (an older reflection library that I wrote was based upon such a system). The first challenge is the location of the global variable, how does this interact with loading shared libraries. Then there is the issue of when does this get populated. You need to initialize a static variable for each type which means that the macro would need to be in the CPP file.... I believe I saw a trick for getting static initialization of variables in a header file via typedef of a template that contains the static initializer. I have not been able to find that trick again. One last detail is that any code that is not referenced is stripped by the static linker so even if you go through all of the trouble to create these static variables, they will be optimized out and never registered with the global factory unless you call a method in the object file that contains the static variable that registers your type with the factory. If you add such a feature to the existing REFLECT() macro then it will result in users paying for something they do not necessarily want. Any solution to this problem will need to be a separate macro, BOOST_REFLECT_REGISTER(TYPE) to register the type with the factory. Perhaps, it would also include information regarding available constructors... Perhaps the REGISTER macro could include the more general REFLECT macro so you get down to only needing one macro. This will only work if we can find a way to initialize static variables in the header. Perhaps you were simply looking to register parsers and not constructors.

Would you be interested if I worked on patch to do this ?

It is a git repo... fork it and send me a pull request with anything you come up with. If we can agree on a solution then I will be more than happy to integrate it with my library. Thanks again for your interest and feedback!