Library Proposal: Reflection for C++

Hi, We would like to propose a library and tools which enhance C++ with reflection capabilities. The package, "Reflex", provides a library for runtime introspection and interaction with C++ "constructs" such as objects, types, scopes, members, templated types, ... The library has no external dependencies and was developed with the ISO/IEC standard for C++ as guideline. Production of dictionary information of arbitrary C++ definitions is done via a Python script (genreflex), in a non-intrusive way, using an external tool (gccxml - www.gccxml.org). The generated dictionary source code can be compiled into a shared library, dynamically loaded and accessed via an API providing the introspection information. You can find more information at http://cern.ch/reflex Below you may find an example code snippet which exposes a subset of the functionality. The four different parts show 1 - the definition of the example classes 2 - how to generate the dictionary information 3 - example on introspection (re-generate classes from the dictionary information) 4 - examples on interaction (get/set data members, invoke functions, con/destruct objects) Cheers Stefan // // 1. original class Foo.h // namespace zot { struct foo_base { public: foo_base() : fBar(4711) {} ~foo_base() {} protected: int fBar; }; class foo : public foo_base { public: int bar(); void set_bar(int i); void set_bar(float f); void operator ++ (); }; inline int foo::bar() { return fBar; } inline void foo::set_bar(float f) { fBar = int(f); } inline void foo::set_bar(int i) { fBar = i; } inline void foo::operator ++ () { ++fBar; } } // namespace zot // // 2. Run python script on Foo.h generating dictionary source code // (Foo_rflx.cpp) and produce a shared library libFooRflx.so // genreflex Foo.h --gccxmlpath=/dir/to/gccxml/bin (e.g.) g++ -shared -o libFooRflx.so Foo_rflx.cpp -I${REFLEXROOT}/ include -L${REFLEXROOT}/lib -lReflex // // 3. + 4. example code for introspection and interaction // #include "Reflex/Reflex.h" using namespace std; using namespace ROOT::Reflex; enum Visibility { Public, Protected, Private }; void generate_class_decl( const Type & cl, const string & indent ) { // ... base class declarations if ( cl.BaseSize()) { for ( Base_Iterator b = cl.Base_Begin(); b != cl.Base_End(); ++b) generate_class_decl((*b).ToType(), indent); } cout << indent << "class " << cl.Name(); // ... bases if ( cl.BaseSize()) { cout << " : " ; for ( Base_Iterator b = cl.Base_Begin(); b != cl.Base_End(); + +b ) { if ( (*b).IsVirtual() ) cout << "virtual "; if ( (*b).IsPublic() ) cout << "public "; if ( (*b).IsProtected() ) cout << "protected "; if ( (*b).IsPrivate() ) cout << "private "; cout << (*b).ToType().Name(SCOPED); if ( b != cl.Base_End()-1 ) cout << ", "; } } cout << " {" << endl; Visibility vis = Private; // ... function members for ( Member_Iterator f = cl.FunctionMember_Begin(); f != cl.FunctionMember_End(); ++f ) { if ( ! (*f).IsArtificial()) { if ( (*f).IsPublic() && vis != Public ) { cout << indent << "public:" << endl; vis = Public; } else if ( (*f).IsProtected() && vis != Protected ) { cout << indent << "protected:" << endl; vis = Protected; } else if ( (*f).IsPrivate() && vis != Private ) { cout << indent << "private:" << endl; vis = Private; } Type ft = (*f).TypeOf(); cout << indent + " "; if ( ! (*f).IsConstructor() && !(*f).IsDestructor() ) cout << ft.ReturnType().Name(SCOPED) << " "; if ( (*f).IsOperator() ) cout << "operator "; cout << (*f).Name() << " ("; if ( ft.FunctionParameterSize() ) { for ( size_t p = 0 ; p < ft.FunctionParameterSize(); p++ ) { cout << ft.FunctionParameterAt(p).Name(SCOPED|QUALIFIED); if ( (*f).FunctionParameterNameAt(p).length() ) cout << " " << (*f).FunctionParameterNameAt(p); if ( (*f).FunctionParameterDefaultAt(p).length() ) cout << " = " << (*f).FunctionParameterDefaultAt(p); if ( p != ft.FunctionParameterSize()-1 ) cout << ", "; } } cout << ");" << endl; } } // ... data members for ( Member_Iterator d = cl.DataMember_Begin(); d != cl.DataMember_End(); ++d ) { if ( (*d).IsPublic() && vis != Public ) { cout << indent << "public:" << endl; vis = Public; } else if ( (*d).IsProtected() && vis != Protected ) { cout << indent << "protected:" << endl; vis = Protected; } else if ( (*d).IsPrivate() && vis != Private ) { cout << indent << "private:" << endl; vis = Private; } cout << indent + " " << (*d).TypeOf().Name(SCOPED) << " " << (*d).Name() << ";" << endl; } cout << indent << "};" << endl; } void generate_class(const Type & ty) { string indent = ""; Scope sc = ty.DeclaringScope(); // ... declaring scope if ( ! sc.IsTopScope() ) { if (sc.IsNamespace()) cout << "namespace "; else if (sc.IsClass()) cout << "class "; cout << sc.Name() << " {" << endl; indent += " "; } generate_class_decl(ty, indent); if ( ! sc.IsTopScope() ) { cout << "}" << endl; if (sc.IsClass()) cout << ";"; } } int main() { // load the dictionary information void * v = dlopen("libFooRflx.so", RTLD_LAZY); // get meta information of type Foo Type fooType = Type::ByName("zot::foo"); // check if the type is valid if (fooType) { // // 3. Introspection // // generate declarations for foo generate_class(fooType); // // 4. Interaction // // update the information for inherited members of class foo // this will be automatic in the future fooType.UpdateMembers(); // construct an object of type Foo Object fooObj = fooType.Construct(); // get the value of the data member (i.e. 4711) int val = Object_Cast<int>(fooObj.Get("fBar")); // set the data member to 4712 fooObj.Set("fBar",++val); // get the data member again (i.e. 4712) val = Object_Cast<int>(fooObj.Get("fBar")); // call function setBar with value 4713 fooObj.Invoke("set_bar",Type::ByName("void (int)"), ++val); // call operator ++ to increase fBar by one fooObj.Invoke("operator++"); // call bar getter and cast the output to int (i.e. 4714) val = Object_Cast<int>(fooObj.Invoke("bar")); // delete the Foo object fooObj.Destruct(); } return 0; } /* // // the output of the introspection part is // namespace zot { class foo_base { public: foo_base (); ~foo_base (); protected: int fBar; }; class foo : public zot::foo_base { public: int bar (); void set_bar (int i); void set_bar (float f); void operator operator++ (); }; } // // // */ -- Stefan Roiser CERN, PH Department CH - 1211 Geneva 23 Mob:+41 76 487 5334 Tel:+41 22 767 4838 Fax:+41 22 767 9425

Stefan Roiser wrote:
Hi,
We would like to propose a library and tools which enhance C++ with reflection capabilities. The package, "Reflex", provides a library for runtime introspection and interaction with C++ "constructs" such as objects, types, scopes, members, templated types, ... The library has no external dependencies and was developed with the ISO/IEC standard for C++ as guideline.
Production of dictionary information of arbitrary C++ definitions is done via a Python script (genreflex), in a non-intrusive way, using an external tool (gccxml - www.gccxml.org). The generated dictionary source code can be compiled into a shared library, dynamically loaded and accessed via an API providing the introspection information.
You can find more information at
You need to provide an easy way in which this library can be built and used at least under Linux and Windows. I have found the information on this sight totally deficient in regards to creating a working version under Windows. Your library may be excellent but unless you allow everyone to see if it is so or not, it is not a viable alternative for all end-users.

Edward Diener <eddielee <at> tropicsoft.com> writes:
Stefan Roiser wrote:
You can find more information at
[...]
I have found the information on this sight totally deficient in regards to creating a working version under Windows. Your library may be excellent but unless you allow everyone to see if it is so or not, it is not a viable alternative for all end-users.
For the *nix world the library and tools should be buildable with the attached autotools. For windows I will upload a VC71 solutions file to the web page. You may try to build it on your own. There are no external dependencies so compiling every .cxx file in src and linking it to a dll should be enough Cheers Stefan
_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost

Stefan Roiser wrote:
Edward Diener <eddielee <at> tropicsoft.com> writes:
You can find more information at
http://cern.ch/reflex [...] I have found the information on
Stefan Roiser wrote: this sight totally deficient in regards to creating a working version under Windows. Your library may be excellent but unless you allow everyone to see if it is so or not, it is not a viable alternative for all end-users.
For the *nix world the library and tools should be buildable with the attached autotools. For windows I will upload a VC71 solutions file to the web page.
I look forward to seeing this for Windows.

On 30 Jan 2006, at 16:13, Edward Diener wrote:
Stefan Roiser wrote:
Edward Diener <eddielee <at> tropicsoft.com> writes:
You can find more information at
http://cern.ch/reflex [...] I have found the information on
Stefan Roiser wrote: this sight totally deficient in regards to creating a working version under Windows. Your library may be excellent but unless you allow everyone to see if it is so or not, it is not a viable alternative for all end-users.
For the *nix world the library and tools should be buildable with the attached autotools. For windows I will upload a VC71 solutions file to the web page.
I look forward to seeing this for Windows.
I have uploaded a VC71 project solution file to the web page at http://seal-reflex.web.cern.ch/seal-reflex/releases.html This should build a Reflex.dll. Sorry for the delay but I first had to find a way how to export the symbols (this is done with our build tools by default). The solution now is to export every symbol. This is too much and I probably should do something similar like boost choosing the proper set of symbols with the decl_export/import for windows. To produce dictionary sources you will need to do python python/genreflex/genreflex.py MyHeader.h --gccxmlpath=c:/my/ path/to/gccxml/bin and then compile the resulting MyHeader_rflx.cpp into a shareable library and dynamically load it into your application (no symbols need to be exported). If you discover problems please let me know. Cheers Stefan -- Stefan Roiser CERN, PH Department CH - 1211 Geneva 23 Mob:+41 76 487 5334 Tel:+41 22 767 4838 Fax:+41 22 767 9425

Stefan Roiser wrote:
On 30 Jan 2006, at 16:13, Edward Diener wrote:
Stefan Roiser wrote:
Edward Diener <eddielee <at> tropicsoft.com> writes:
You can find more information at
http://cern.ch/reflex [...] I have found the information on
Stefan Roiser wrote: this sight totally deficient in regards to creating a working version under Windows. Your library may be excellent but unless you allow everyone to see if it is so or not, it is not a viable alternative for all end-users. For the *nix world the library and tools should be buildable with the attached autotools. For windows I will upload a VC71 solutions file to the web page. I look forward to seeing this for Windows.
I have uploaded a VC71 project solution file to the web page at
http://seal-reflex.web.cern.ch/seal-reflex/releases.html
This should build a Reflex.dll. Sorry for the delay but I first had to find a way how to export the symbols (this is done with our build tools by default). The solution now is to export every symbol. This is too much and I probably should do something similar like boost choosing the proper set of symbols with the decl_export/import for windows.
To produce dictionary sources you will need to do
python python/genreflex/genreflex.py MyHeader.h --gccxmlpath=c:/my/ path/to/gccxml/bin
and then compile the resulting MyHeader_rflx.cpp into a shareable library and dynamically load it into your application (no symbols need to be exported).
If you discover problems please let me know.
Thanks, I have it and will be looking at it. I also encourage you to put the information about this on your web site for other VC++ developers to use also in order to try Reflex on Windows. I am very much interested in a C++ run-time reflection system and, after I have worked with Reflex will gladly communicate my ideas.

Stefan Roiser <stefan.roiser@cern.ch> writes:
Hi,
We would like to propose a library and tools which enhance C++ with reflection capabilities. The package, "Reflex", provides a library for runtime introspection and interaction with C++ "constructs" such as objects, types, scopes, members, templated types, ... The library has no external dependencies and was developed with the ISO/IEC standard for C++ as guideline.
Production of dictionary information of arbitrary C++ definitions is done via a Python script (genreflex), in a non-intrusive way, using an external tool (gccxml - www.gccxml.org). The generated dictionary source code can be compiled into a shared library, dynamically loaded and accessed via an API providing the introspection information.
You can find more information at
Below you may find an example code snippet which exposes a subset of the functionality. The four different parts show 1 - the definition of the example classes 2 - how to generate the dictionary information 3 - example on introspection (re-generate classes from the dictionary information) 4 - examples on interaction (get/set data members, invoke functions, con/destruct objects)
How does this work relate or compare to http://www.language-binding.net/pygccxml/pygccxml.html? The latter already has some traction in the Boost community because of its use in http://www.language-binding.net/pyplusplus/pyplusplus.html -- Dave Abrahams Boost Consulting www.boost-consulting.com

David Abrahams <dave <at> boost-consulting.com> writes:
Stefan Roiser <stefan.roiser <at> cern.ch> writes:
Hi,
We would like to propose a library and tools which enhance C++ with reflection capabilities. The package, "Reflex", provides a library for runtime introspection and interaction with C++ "constructs" such as objects, types, scopes, members, templated types, ... The library has no external dependencies and was developed with the ISO/IEC standard for C++ as guideline.
Production of dictionary information of arbitrary C++ definitions is done via a Python script (genreflex), in a non-intrusive way, using an external tool (gccxml - www.gccxml.org). The generated dictionary source code can be compiled into a shared library, dynamically loaded and accessed via an API providing the introspection information.
You can find more information at
Below you may find an example code snippet which exposes a subset of the functionality. The four different parts show 1 - the definition of the example classes 2 - how to generate the dictionary information 3 - example on introspection (re-generate classes from the dictionary information) 4 - examples on interaction (get/set data members, invoke functions, con/destruct objects)
How does this work relate or compare to http://www.language-binding.net/pygccxml/pygccxml.html?
The latter already has some traction in the Boost community because of its use in http://www.language-binding.net/pyplusplus/pyplusplus.html
I had already an off-list discussion with the developers of pygccxml. I have been using it for private scripts and found it very useful. The Reflex package provides a python script (genreflex.py) which produces dictionary source code for C++ definitions. In order to do this it invokes gccxml on these definitions and parses it's output. At the moment genreflex.py uses it's own data structures. In the future it could be useful to switch to pygccxml. Unfortunately at the moment we don't have the manpower to do this. Cheers Stefan

Stefan Roiser <stefan.roiser <at> cern.ch> writes:
Hi,
We would like to propose a library and tools which enhance C++ with reflection capabilities. The package, "Reflex", provides a library for runtime introspection and interaction with C++ "constructs" such as objects, types, scopes, members, templated types, ... The library has no external dependencies and was developed with the ISO/IEC standard for C++ as guideline.
It sounds pretty interesting:) I extremely hoped that C++ would have reflection at, maybe, C++0x or so. So I wish your library can make its way into boost, and, finally, into C++ standard library. BTW, to me, it seems the user-interface of the library needs some further refinement though, and there's quite a lack of documents. Regards.

pongba <pongba <at> gmail.com> writes:
Stefan Roiser <stefan.roiser <at> cern.ch> writes:
We would like to propose a library and tools which enhance C++ with reflection capabilities. The package, "Reflex", provides a library for runtime introspection and interaction with C++ "constructs" such as objects, types, scopes, members, templated types, ...
It sounds pretty interesting:) I extremely hoped that C++ would have reflection at, maybe, C++0x or so. So I wish your library can make its way into boost, and, finally, into C++ standard library.
BTW, to me, it seems the user-interface of the library needs some further refinement though, and there's quite a lack of documents.
Thx, I will be very interested to receive comments on the library. There are some examples on the web-page to give a look and feel of how it works. I have added doxygen documentation for the API classes at http://seal.web.cern.ch/seal/documents/dictionary/reflex/doxygen/html/index....

Stefan Roiser wrote:
Hi,
We would like to propose a library and tools which enhance C++ with reflection capabilities. The package, "Reflex", provides a library for runtime introspection and interaction with C++ "constructs" such as objects, types, scopes, members, templated types, ... The library has no external dependencies and was developed with the ISO/IEC standard for C++ as guideline.
Production of dictionary information of arbitrary C++ definitions is done via a Python script (genreflex), in a non-intrusive way, using an external tool (gccxml - www.gccxml.org). The generated dictionary source code can be compiled into a shared library, dynamically loaded and accessed via an API providing the introspection information.
How does this compare to OpenC++ (http://opencxx.sf.net) ? Also, have you looked into synopsis (http://synopsis.fresco.org) ? It started by using the OpenC++ parser and infrastructure, but has been largely rewritten since then. Regards, Stefan

On 30 Jan 2006, at 15:03, Stefan Seefeld wrote:
Stefan Roiser wrote:
Hi,
We would like to propose a library and tools which enhance C++ with reflection capabilities. The package, "Reflex", provides a library for runtime introspection and interaction with C++ "constructs" such as objects, types, scopes, members, templated types, ... The library has no external dependencies and was developed with the ISO/IEC standard for C++ as guideline.
Production of dictionary information of arbitrary C++ definitions is done via a Python script (genreflex), in a non-intrusive way, using an external tool (gccxml - www.gccxml.org). The generated dictionary source code can be compiled into a shared library, dynamically loaded and accessed via an API providing the introspection information.
How does this compare to OpenC++ (http://opencxx.sf.net) ?
Also, have you looked into synopsis (http://synopsis.fresco.org) ? It started by using the OpenC++ parser and infrastructure, but has been largely rewritten since then.
I was not aware of synopsis. Both projects are very interesting but their scope seems to be different. I understood that both use reflection as a vehicle for their needs while Reflex only concentrates on introspection and interaction with C++. Reflex does also not try to parse C++ definitions itself but leaves this to gccxml (a frontend of gcc) and subsequently processes it's xml output to produce dictionary source code. My guess is, that it is very hard to implement a C++ parser. So with Reflex the dictionary information for any C++ definition which is compilable with gcc, can be produced. What may also be important is, that reflection source code is produced in a non-intrusive way and that interaction with objects from the meta-level is possible. Cheers Stefan -- Stefan Roiser CERN, PH Department CH - 1211 Geneva 23 Mob:+41 76 487 5334 Tel:+41 22 767 4838 Fax:+41 22 767 9425

Stefan Roiser wrote:
I was not aware of synopsis. Both projects are very interesting but their scope seems to be different. I understood that both use reflection as a vehicle for their needs while Reflex only concentrates on introspection and interaction with C++.
Synopsis is meant to be a code-source introspection framework (with the first application being a documentation tool). From what I understand it has about the same scope as Reflex.
Reflex does also not try to parse C++ definitions itself but leaves this to gccxml (a frontend of gcc) and subsequently processes it's xml output to produce dictionary source code. My guess is, that it is very hard to implement a C++ parser. So with Reflex the dictionary information for any C++ definition which is compilable with gcc, can be produced.
Right, though gccxml has some drawbacks, too.
What may also be important is, that reflection source code is produced in a non-intrusive way and that interaction with objects from the meta-level is possible.
That is true for synopsis, too. FWIW, Stefan

On 30 Jan 2006, at 17:08, Stefan Seefeld wrote:
Reflex does also not try to parse C++ definitions itself but leaves this to gccxml (a frontend of gcc) and subsequently processes it's xml output to produce dictionary source code. My guess is, that it is very hard to implement a C++ parser. So with Reflex the dictionary information for any C++ definition which is compilable with gcc, can be produced.
Right, though gccxml has some drawbacks, too.
We have fortunately not found problems yet and we generated dictionaries for a lot of user code in our environment and also "external" libraries. These dictionaries are then e.g. further used for persistence of objects.
What may also be important is, that reflection source code is produced in a non-intrusive way and that interaction with objects from the meta-level is possible.
That is true for synopsis, too.
With interaction with objects I meant e.g. through the meta-level instantiate a class, get/set the data member values, call a function on that instance, retrieve the function's return value and so on. I may have missed that, but I couldn't find examples on the synopsis page for this functionality. Reflex code for this would look something like Type t = Type::ByName("foo"); Object instance = t.Construct(); Object ret_obj = instance.Invoke("myFunction"); Cheers Stefan -- Stefan Roiser CERN, PH Department CH - 1211 Geneva 23 Mob:+41 76 487 5334 Tel:+41 22 767 4838 Fax:+41 22 767 9425

Stefan Roiser wrote:
What may also be important is, that reflection source code is produced in a non-intrusive way and that interaction with objects from the meta-level is possible.
That is true for synopsis, too.
With interaction with objects I meant e.g. through the meta-level instantiate a class, get/set the data member values, call a function on that instance, retrieve the function's return value and so on. I may have missed that, but I couldn't find examples on the synopsis page for this functionality.
You are right, that's not possible. I was thinking of interactions with the objects representing the parse tree, syntax tree, symbol table, etc. Interacting with the running code may involve quite a bit of work that, at least when taken in its most general form, requires knowledge that a parser (or compiler frontend) can not have just from introspecting the code. Synopsis provides C++ and python interfaces to the internal representations, and so my hope is that higher level tools, even ones that compile and load the introspected code, can easily be built (e.g. scripted) with them.
Reflex code for this would look something like
Type t = Type::ByName("foo"); Object instance = t.Construct();
That clearly only works if 'foo' has a default constructor, right ? Even then, it may involve some other resources to be set up. So, what knowledge of the code being invoked that way do you require ? Who would provide the implementation for 'foo', who would provide the above lines, and who would finally execute them ?
Object ret_obj = instance.Invoke("myFunction");
Regards, Stefan

On 30 Jan 2006, at 23:55, Stefan Seefeld wrote:
Stefan Roiser wrote:
Reflex code for this would look something like
Type t = Type::ByName("foo"); Object instance = t.Construct();
That clearly only works if 'foo' has a default constructor, right ?
You are right, the above code snippet has a lot of assumptions (e.g. we found a dictionary for type foo, foo has a default constructor etc.), please see below for a bit more safe implementation.
Even then, it may involve some other resources to be set up. So, what knowledge of the code being invoked that way do you require ?
Reflex would need a dictionary for class "foo" (generated by genreflex.py with gccxml). So something like python genreflex.py foo.h --gccxmlpath=/path/to/gccxml/bin will generate a file foo_rflx.cpp with the dictionary information about class foo. This file will contain all the information necessary to introspect/ interact with foo and can be compiled into a shared library libfoo.so
Who would provide the implementation for 'foo', who would provide the above lines, and who would finally execute them ?
I'm assuming that class foo is a class for which the dictionary is generated and subsequently a user want's to introspect/interact with its dictionary information. So a bit more safe implementation of the lines above would be // load dynamically library libfoo.so Type t = Type::ByName("foo"); if (t && t.IsClass()) { // we found the dictionary information for "foo" and it's a class // let's try to call the default constructor // Construct has several default arguments, by default it will try to invoke the default constructor Object instance = t.Construct(); if (instance) { // we have a valid instance (memory allocated, type associated) // we have constructed an instance through the default constructor } else { // this class does not have a default constructor // let's loop over all constructors and look for one which takes e.g. one int as argument for (Member_iterator mi = t.FunctionMember_Begin(); mi != t.FunctionMember_End(); ++mi) { if ((*mi).IsConstructor()) { Type constType = (*mi).TypeOf(); if (constType.FunctionParameterSize() == 1 && constType.FunctionParameterAt(0).TypeOf().Name() == "int") { // let's invoke this constructor with one argument (this has to be passed in a vector<void*>) int arg0 = 111; std::vector<void*> args; args.push_back(&arg0); instance = t.Construct(constType,args); ....... Cheers Stefan -- Stefan Roiser CERN, PH Department CH - 1211 Geneva 23 Mob:+41 76 487 5334 Tel:+41 22 767 4838 Fax:+41 22 767 9425

Stefan Roiser <stefan.roiser <at> cern.ch> writes:
On 30 Jan 2006, at 23:55, Stefan Seefeld wrote:
Stefan Roiser wrote:
Reflex code for this would look something like
Type t = Type::ByName("foo"); Object instance = t.Construct();
That clearly only works if 'foo' has a default constructor, right ?
Just so you know, I'm strongly wondering if you could implement a serialization library(e.g. boost::serialization) using Reflex. No doubt it should have that capability. Not only would this be a good example that illustrates the power of your framework, it would contribute a much-easier-to-use serialization library as well, considering that boost::serialization is a little inconvenient to use. Regards.

On 31 Jan 2006, at 15:44, pongba wrote:
Stefan Roiser <stefan.roiser <at> cern.ch> writes:
On 30 Jan 2006, at 23:55, Stefan Seefeld wrote:
Stefan Roiser wrote:
Reflex code for this would look something like
Type t = Type::ByName("foo"); Object instance = t.Construct();
That clearly only works if 'foo' has a default constructor, right ?
Just so you know, I'm strongly wondering if you could implement a serialization library(e.g. boost::serialization) using Reflex. No doubt it should have that capability. Not only would this be a good example that illustrates the power of your framework, it would contribute a much-easier-to-use serialization library as well, considering that boost::serialization is a little inconvenient to use.
I will be very happy if boost::serialization shows interest in Reflex. But in fact such projects already exist at CERN :-) - The POOL project does persistence of C++ objects and uses Reflex for this purpose. You may have a look at http://pool.cern.ch/ for details. - The ROOT project, among other things, also does persistence of C++ objects. We are currently working on changing the underlying reflection system for ROOT to switch to Reflex. More infos at http:// root.cern.ch -- Stefan Roiser CERN, PH Department CH - 1211 Geneva 23 Mob:+41 76 487 5334 Tel:+41 22 767 4838 Fax:+41 22 767 9425

Stefan Roiser <stefan.roiser@cern.ch> writes:
On 30 Jan 2006, at 17:08, Stefan Seefeld wrote:
With interaction with objects I meant e.g. through the meta-level instantiate a class, get/set the data member values, call a function on that instance, retrieve the function's return value and so on. I may have missed that, but I couldn't find examples on the synopsis page for this functionality.
Reflex code for this would look something like
Type t = Type::ByName("foo"); Object instance = t.Construct(); Object ret_obj = instance.Invoke("myFunction");
Very interesting. Where do you get the object code for foo::myFunction? Or have you implemented a full C++ interpreter? Have you considered something like the object interface of Boost.Python? Type t = Type::ByName("foo"): Object instance = t(); Object ret_obj = instance.member("myFunction")(); Then you'd also have, e.g. Object enlarged = instance & 5; -- Dave Abrahams Boost Consulting www.boost-consulting.com

On 31 Jan 2006, at 19:40, David Abrahams wrote:
Stefan Roiser <stefan.roiser@cern.ch> writes:
On 30 Jan 2006, at 17:08, Stefan Seefeld wrote:
With interaction with objects I meant e.g. through the meta-level instantiate a class, get/set the data member values, call a function on that instance, retrieve the function's return value and so on. I may have missed that, but I couldn't find examples on the synopsis page for this functionality.
Reflex code for this would look something like
Type t = Type::ByName("foo"); Object instance = t.Construct(); Object ret_obj = instance.Invoke("myFunction");
Very interesting. Where do you get the object code for foo::myFunction? Or have you implemented a full C++ interpreter?
foo::myFunction will be invoked through a stub function which is part of the dictionary library for class foo. A dictionary library may be dynamically loaded in a program. The source code for this dictionary library is produced in a non- intrusive way with a python script (genreflex.py). Besides the stub functions this dictionary source code contains the information about all types, scopes, members one wants to generate the reflection information for. To give you an idea, typically this automatically produced code would look like (this is part of the code for the example class of my first mail) static void* constructor_3558( void* mem, const std::vector<void*>& arg, void*) { return ::new(mem) ::zot::foo(*(const ::zot::foo*)arg[0]); } static void* constructor_3559( void* mem, const std::vector<void*>&, void*) { return ::new(mem) ::zot::foo(); } static void* destructor_3560(void * o, const std::vector<void*>&, void *) { ((::zot::foo*)o)->~foo(); return 0; } static void* method_3561( void* o, const std::vector<void*>&, void*) { static int ret; ret = ((::zot::foo*)o)->bar(); return &ret; } ...... void __zot__foo_dict() { ClassBuilder("zot::foo", typeid(zot::foo), sizeof(zot::foo), PUBLIC | CLASS) .AddBase(type_2263, BaseOffset< ::zot::foo, ::zot::foo_base >::Get (), VIRTUAL | PUBLIC) .AddFunctionMember(FunctionTypeBuilder(type_void, type_4260), "foo", constructor_3558, 0, "_ctor_arg", PUBLIC | ARTIFICIAL | CONSTRUCTOR) .AddFunctionMember(FunctionTypeBuilder(type_void), "foo", constructor_3559, 0, 0, PUBLIC | ARTIFICIAL | CONSTRUCTOR) .AddFunctionMember(FunctionTypeBuilder(type_void), "~foo", destructor_3560, 0, 0, PUBLIC | ARTIFICIAL | DESTRUCTOR ) .AddFunctionMember(FunctionTypeBuilder(type_9), "bar", method_3561, 0, 0, PUBLIC) ......
Have you considered something like the object interface of Boost.Python?
Type t = Type::ByName("foo"): Object instance = t(); Object ret_obj = instance.member("myFunction")();
Then you'd also have, e.g.
Object enlarged = instance & 5;
To be frank - no, but this is a very good idea. Thx for the input!
-- Dave Abrahams Boost Consulting www.boost-consulting.com
_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/ listinfo.cgi/boost
-- Stefan Roiser CERN, PH Department CH - 1211 Geneva 23 Mob:+41 76 487 5334 Tel:+41 22 767 4838 Fax:+41 22 767 9425

foo::myFunction will be invoked through a stub function which is part of the dictionary library for class foo. A dictionary library may be dynamically loaded in a program.
The source code for this dictionary library is produced in a non- intrusive way with a python script (genreflex.py). Besides the stub functions this dictionary source code contains the information about all types, scopes, members one wants to generate the reflection information for.
This sounds similar to another CERN project: Cint (http://root.cern.ch/root/Cint.html), the C/C++ interpreter. Do they share code? This reflection method seems interesting (although a native one would be better no doubt). In addition to your library I would encourage Cint guys to present the library to Boost (for example, they could use the Wave preprocessor for complex C/C++ instead of writing their own). A C/C++ interpreter would be killer for Boost (having also native<->interpreted transitions). Regards, Ion

On 1 Feb 2006, at 07:06, Ion GaztaƱaga wrote:
foo::myFunction will be invoked through a stub function which is part of the dictionary library for class foo. A dictionary library may be dynamically loaded in a program.
The source code for this dictionary library is produced in a non- intrusive way with a python script (genreflex.py). Besides the stub functions this dictionary source code contains the information about all types, scopes, members one wants to generate the reflection information for.
This sounds similar to another CERN project: Cint (http://root.cern.ch/root/Cint.html), the C/C++ interpreter. Do they share code?
As you may see from the Reflex web page (cern.ch/reflex) Reflex has moved recently into the ROOT project. ROOT currently uses Cint as dictionary and interpreter. We are currently working full steam on a merge of Reflex with Cint. This means that Cint soon will be able to use the Reflex dictionary system.
This reflection method seems interesting (although a native one would be better no doubt).
I very much agree with you, native reflection for C++ would be the best thing we could have.
In addition to your library I would encourage Cint guys to present the library to Boost (for example, they could use the Wave preprocessor for complex C/C++ instead of writing their own). A C/C++ interpreter would be killer for Boost (having also native<->interpreted transitions).
Thanks for the information I will discuss this with the Cint developers. -- Stefan Roiser CERN, PH Department CH - 1211 Geneva 23 Mob:+41 76 487 5334 Tel:+41 22 767 4838 Fax:+41 22 767 9425

Stefan Roiser <stefan.roiser@cern.ch> writes:
On 31 Jan 2006, at 19:40, David Abrahams wrote:
Stefan Roiser <stefan.roiser@cern.ch> writes:
On 30 Jan 2006, at 17:08, Stefan Seefeld wrote:
With interaction with objects I meant e.g. through the meta-level instantiate a class, get/set the data member values, call a function on that instance, retrieve the function's return value and so on. I may have missed that, but I couldn't find examples on the synopsis page for this functionality.
Reflex code for this would look something like
Type t = Type::ByName("foo"); Object instance = t.Construct(); Object ret_obj = instance.Invoke("myFunction");
Very interesting. Where do you get the object code for foo::myFunction? Or have you implemented a full C++ interpreter?
foo::myFunction will be invoked through a stub function which is part of the dictionary library for class foo. A dictionary library may be dynamically loaded in a program.
Soooo.... what about templates? Do you have to do something to manually instantiate them? Or do you do some kind of on-the-fly compilation? -- Dave Abrahams Boost Consulting www.boost-consulting.com

David Abrahams wrote:
Stefan Roiser <stefan.roiser@cern.ch> writes:
On 31 Jan 2006, at 19:40, David Abrahams wrote:
Stefan Roiser <stefan.roiser@cern.ch> writes:
On 30 Jan 2006, at 17:08, Stefan Seefeld wrote:
With interaction with objects I meant e.g. through the meta-level instantiate a class, get/set the data member values, call a function on that instance, retrieve the function's return value and so on. I may have missed that, but I couldn't find examples on the synopsis page for this functionality.
Reflex code for this would look something like
Type t = Type::ByName("foo"); Object instance = t.Construct(); Object ret_obj = instance.Invoke("myFunction"); Very interesting. Where do you get the object code for foo::myFunction? Or have you implemented a full C++ interpreter? foo::myFunction will be invoked through a stub function which is part of the dictionary library for class foo. A dictionary library may be dynamically loaded in a program.
Soooo.... what about templates? Do you have to do something to manually instantiate them? Or do you do some kind of on-the-fly compilation?
Since templates are not types, I would leave out the instantiation of templates into types, which is a compile-time mechanism, from a run-time reflection system, and concentrate instead on finding information about types ( and objects ), and invoking objects from types. Of course an already instantiated template class or template function is a type, and should be subject to run-time reflection like any other type.

Edward Diener <eddielee@tropicsoft.com> writes:
David Abrahams wrote:
Stefan Roiser <stefan.roiser@cern.ch> writes:
On 31 Jan 2006, at 19:40, David Abrahams wrote:
Stefan Roiser <stefan.roiser@cern.ch> writes:
On 30 Jan 2006, at 17:08, Stefan Seefeld wrote:
With interaction with objects I meant e.g. through the meta-level instantiate a class, get/set the data member values, call a function on that instance, retrieve the function's return value and so on. I may have missed that, but I couldn't find examples on the synopsis page for this functionality.
Reflex code for this would look something like
Type t = Type::ByName("foo"); Object instance = t.Construct(); Object ret_obj = instance.Invoke("myFunction"); Very interesting. Where do you get the object code for foo::myFunction? Or have you implemented a full C++ interpreter? foo::myFunction will be invoked through a stub function which is part of the dictionary library for class foo. A dictionary library may be dynamically loaded in a program.
Soooo.... what about templates? Do you have to do something to manually instantiate them? Or do you do some kind of on-the-fly compilation?
Since templates are not types, I would leave out the instantiation of templates into types, which is a compile-time mechanism, from a run-time reflection system
That presumes that the only thing worth reflecting about are types. I of course want reflection into templates (and not just class templates, either). But -- no offense intended of course -- unless you're part of this effort, I'm not really asking what you would do. I want to know what the Cern people are doing and are planning to do. -- Dave Abrahams Boost Consulting www.boost-consulting.com

David Abrahams wrote:
Edward Diener <eddielee@tropicsoft.com> writes:
David Abrahams wrote:
Stefan Roiser <stefan.roiser@cern.ch> writes:
On 31 Jan 2006, at 19:40, David Abrahams wrote:
Stefan Roiser <stefan.roiser@cern.ch> writes:
On 30 Jan 2006, at 17:08, Stefan Seefeld wrote:
With interaction with objects I meant e.g. through the meta-level instantiate a class, get/set the data member values, call a function on that instance, retrieve the function's return value and so on. I may have missed that, but I couldn't find examples on the synopsis page for this functionality.
Reflex code for this would look something like
Type t = Type::ByName("foo"); Object instance = t.Construct(); Object ret_obj = instance.Invoke("myFunction"); Very interesting. Where do you get the object code for foo::myFunction? Or have you implemented a full C++ interpreter? foo::myFunction will be invoked through a stub function which is part of the dictionary library for class foo. A dictionary library may be dynamically loaded in a program. Soooo.... what about templates? Do you have to do something to manually instantiate them? Or do you do some kind of on-the-fly compilation? Since templates are not types, I would leave out the instantiation of templates into types, which is a compile-time mechanism, from a run-time reflection system
That presumes that the only thing worth reflecting about are types. I of course want reflection into templates (and not just class templates, either). But -- no offense intended of course --
No offense taken.
unless you're part of this effort, I'm not really asking what you would do. I want to know what the Cern people are doing and are planning to do.
The Cern people will have to speak for themselves. I was not intending to speak for Reflex but giving my own opinions on the matter. However my undestanding, from the docs, of Reflex is that it is a run-time reflection mechanism, even though its dictionary is built prior to run-time. If that is so that it is a run-time mechanism, I do not think that reflecting about template classes and/or template functions should be a run-time task, since at run-time instantiated template classes and/or template functions become types. Nor do I understand how reflection about templates at run-time could give any advantage to a programmer. I think that there may be a very good case for a compile-time reflection mechanism which could reflect templates, and which would be available to an expert template metaprogrammer like yourself, and many others doing template programming. at compile-time. But I can not see how this would be done unless built-in to the language or particular compilers themselves as an extension to the language. But that is much more your province than mine, so you may have much better ideas how this could be done than I do.

On 2 Feb 2006, at 01:51, Edward Diener wrote:
David Abrahams wrote:
Edward Diener <eddielee@tropicsoft.com> writes:
David Abrahams wrote:
Stefan Roiser <stefan.roiser@cern.ch> writes:
On 31 Jan 2006, at 19:40, David Abrahams wrote:
Stefan Roiser <stefan.roiser@cern.ch> writes:
> On 30 Jan 2006, at 17:08, Stefan Seefeld wrote: > > With interaction with objects I meant e.g. through the meta- > level > instantiate a class, get/set the data member values, call a > function > on that instance, retrieve the function's return value and so > on. I > may have missed that, but I couldn't find examples on the > synopsis > page for this functionality. > > Reflex code for this would look something like > > Type t = Type::ByName("foo"); > Object instance = t.Construct(); > Object ret_obj = instance.Invoke("myFunction"); Very interesting. Where do you get the object code for foo::myFunction? Or have you implemented a full C++ interpreter? foo::myFunction will be invoked through a stub function which is part of the dictionary library for class foo. A dictionary library may be dynamically loaded in a program. Soooo.... what about templates? Do you have to do something to manually instantiate them? Or do you do some kind of on-the-fly compilation? Since templates are not types, I would leave out the instantiation of templates into types, which is a compile-time mechanism, from a run-time reflection system
That presumes that the only thing worth reflecting about are types. I of course want reflection into templates (and not just class templates, either). But -- no offense intended of course --
No offense taken.
unless you're part of this effort, I'm not really asking what you would do. I want to know what the Cern people are doing and are planning to do.
The Cern people will have to speak for themselves.
Reflex does provide reflection information for template instances (classes, functions) and template families. An instantiated template class is a Reflex::Type, an instantiated template function is a Reflex::Member. There are two additional classes in the API (TypeTemplate, MemberTemplate) which will allow the user to introspect/interact with a template family (e.g. std::vector<T>), but this information only get's filled once the first template instance in the dictionary source code comes along. The TypeTemplate/MemberTemplate currently allow to loop over template parameters, look at parameter defaults (if any, this also needs some extra handling - gxxml won't give us this info) and loop over the instances of this template currently available in the system. With Reflex our scope was, as also pointed out by Edward, mainly the runtime reflection part. We are using Reflex at the moment for persistence of objects and interactive usage of objects (with language bindings - eg. Python). If I understood correctly your point, I think the two "xxxTemplate" classes are going in the direction of template introspection you intend to do, but you probably want to have more information. But how to retrieve this additional information? Related to this topic is the generation of dictionary information for a template instance. When generating dictionary information we normally select which information we want to generate (we don't want to generate the dictionary for vector<int> n times). So in general we generate it once and provide these "common" dictionary libraries to our users. In order to do that we have devloped a selection mechanism which is passed as an xml description to the dictionary generation step. To generate the dictionary for an vector of int and double would be generated like. MyVector.h ========== #include <vector> namespace { struct _Dict { std::vector<int> m1; std::vector<double> m2; }; } This syntax is currently needed because we use a quite old version of gccxml. In the future only the declaration of the template type will be needed to trigger gccxml to output the information for a template type. selection.xml ============= <lcgdict> <class pattern="std::vector<*>"/> </lcgdict> (yes, "<" and ">" are allowed in the xml descriptions as text for convenience) the syntax for the xml selections is more powerful than this simple example. You may have a look at http://seal-reflex.web.cern.ch/seal- reflex/gendict.html#selection_syntax for details. python genreflex.py MyVector.h -s selection.xml --gccxmplath=/my/path/ to/gccxml/bin The so into a library compiled dictionary from MyVector_rflx.cpp will when loaded into Reflex produce a TypeTemplate "std::vector" which contains a list of instances for the vector<int> and the vector<double> -- Stefan Roiser CERN, PH Department CH - 1211 Geneva 23 Mob:+41 76 487 5334 Tel:+41 22 767 4838 Fax:+41 22 767 9425

Stefan Roiser wrote:
Reflex does provide reflection information for template instances (classes, functions) and template families. An instantiated template class is a Reflex::Type, an instantiated template function is a Reflex::Member. There are two additional classes in the API (TypeTemplate, MemberTemplate) which will allow the user to introspect/interact with a template family (e.g. std::vector<T>), but this information only get's filled once the first template instance in the dictionary source code comes along.
The TypeTemplate/MemberTemplate currently allow to loop over template parameters, look at parameter defaults (if any, this also needs some extra handling - gxxml won't give us this info) and loop over the instances of this template currently available in the system.
I think it is a commendable goal in Reflex to be able to reflect the parameter types for any given template instantiation, since this information may be valuable and usable at run-time. After all, at run-time, information about a template type should be just as accesible regarding finding type information than any other type. One of the pieces of information about a template type is the parameter types which are used to instantiate the template, so run-time reflection should include such information. But I feel that information about the template declaration/definition itself belongs in the realm of a compile-time reflection system, not a run-time reflection system, and can serve no purpose in a run-time reflection system.

Edward Diener <eddielee@tropicsoft.com> writes:
I think it is a commendable goal in Reflex to be able to reflect the parameter types for any given template instantiation, since this information may be valuable and usable at run-time. After all, at run-time, information about a template type should be just as accesible regarding finding type information than any other type. One of the pieces of information about a template type is the parameter types which are used to instantiate the template, so run-time reflection should include such information. But I feel that information about the template declaration/definition itself belongs in the realm of a compile-time reflection system, not a run-time reflection system, and can serve no purpose in a run-time reflection system.
Even if the run-time reflection system is generating code? The boundaries are a lot more fluid than you probably imagine. -- Dave Abrahams Boost Consulting www.boost-consulting.com

David Abrahams wrote:
Even if the run-time reflection system is generating code? The boundaries are a lot more fluid than you probably imagine.
That, however, means integrating a (mostly) complete C++ compiler or interpreter into the reflection library (mostly in that it can interpret/compile an intermediate form instead of real source). Complexity and licensing aside, a compiler is also inherently unportable. It would be a fascinating project, to be sure, but vast. Sebastian Redl

Sebastian Redl wrote:
David Abrahams wrote:
Even if the run-time reflection system is generating code? The boundaries are a lot more fluid than you probably imagine.
That, however, means integrating a (mostly) complete C++ compiler or interpreter into the reflection library (mostly in that it can interpret/compile an intermediate form instead of real source). Complexity and licensing aside, a compiler is also inherently unportable. It would be a fascinating project, to be sure, but vast.
<shameless.plug> What better introduction could I ask for to advertise the synopsis project (http://synopsis.fresco.org) ? :-) While It is far from perfect, it already supports a number of interesting features that border at introspection (i.e. it generates various code representations such as parse tree, symbol table, etc. I would very much appreciate help to work on the missing aspects to complete various analysis passes (inclusive the C++ parser rewrite I'm currently working on). I'm not quite sure what 'inherent' portability issues you anticipate. The framework is written in (std) C++ and python, and the parser itself in pure C++. </shameless.plug> Regards, Stefan

Stefan Seefeld wrote:
I'm not quite sure what 'inherent' portability issues you anticipate. The framework is written in (std) C++ and python, and the parser itself in pure C++.
The inherent portability issues lie at the very end of the back-end, in the code generation. If it is to produce native code, it's unportable. If it produces bytecode, you again need an interpreter, but the native call interface might still be unportable. Sebastian Redl

Sebastian Redl wrote:
Stefan Seefeld wrote:
I'm not quite sure what 'inherent' portability issues you anticipate. The framework is written in (std) C++ and python, and the parser itself in pure C++.
The inherent portability issues lie at the very end of the back-end, in the code generation. If it is to produce native code, it's unportable.
Ah, obviously. I'm only talking about source-to-source translation and other introspection use cases, which 'only' require a front-end. Regards, Stefan

Sebastian Redl <sebastian.redl@getdesigned.at> writes:
David Abrahams wrote:
Even if the run-time reflection system is generating code? The boundaries are a lot more fluid than you probably imagine.
That, however, means integrating a (mostly) complete C++ compiler or interpreter into the reflection library (mostly in that it can interpret/compile an intermediate form instead of real source).
That might be easier than you think. There's always system("c++ whatever.cpp...") There's a really interesting python project that lets you embed C++ directly in your Python function. Remember, Python is fully dynamic, so the types passed could be anything. The C++ gets compiled on-the-fly, as needed, for the different combinations of argument types passed to python. They didn't invent a whole new C++ compiler, they just require you to have one in the system, and they use that. -- Dave Abrahams Boost Consulting www.boost-consulting.com

David Abrahams wrote:
That might be easier than you think. There's always
system("c++ whatever.cpp...")
Still unportable. The MS compiler is called cl, the Borland one bcc. What about different versions of GCC that might be called, and their varying ABIs? How to know the exact object file format produced? What if the application doesn't have write access to a temporary directory to place source and object file in? For that matter, how does the application find out the temporary directory? Yes, it's possible to invoke an installed compiler to produce object code, but it creates a lot of problems on its own. I think the most promising idea would be to combine an existing special representation (e.g. that of Synopsis) with a custom gcc frontend and embed the gcc backend into a library. Intercept the output. A custom linker that is able to link with a running application would probably be needed. Then the only remaining problem is that GCC's license is incompatible with Boost's. Sebastian Redl

Sebastian Redl wrote:
I think the most promising idea would be to combine an existing special representation (e.g. that of Synopsis) with a custom gcc frontend and embed the gcc backend into a library. Intercept the output. A custom linker that is able to link with a running application would probably be needed.
I second what David said: just use the system compiler / linker to produce a DSO and load that into your process. Anything else is going to be much more complex. And, I don't see why you would need Synopsis *and* a custom gcc frontend (such as gccxml). You can generate a parse tree (etc.) with synopsis alone, process it either in C++ or python to generate new code, then call a system (or custom) toolchain to compile that into a loadable module. If you are doing all this in python you don't even need to care about ABI compatibility issues.
Then the only remaining problem is that GCC's license is incompatible with Boost's.
Not a problem if you don't *link* with gcc code (synopsis is distributed under LGPL). Regards, Stefan

Stefan Seefeld wrote:
Sebastian Redl wrote:
I think the most promising idea would be to combine an existing special representation (e.g. that of Synopsis) with a custom gcc frontend and embed the gcc backend into a library. Intercept the output. A custom linker that is able to link with a running application would probably be needed.
I second what David said: just use the system compiler / linker to produce a DSO and load that into your process. Anything else is going to be much more complex. And, I don't see why you would need Synopsis *and* a custom gcc frontend (such as gccxml).
What does Synopsis have that Reflex does not as far as C++ reflection is concerned ? What reflection problem are you trying to solve with whatever Synopsis can do ?

Edward Diener wrote:
What does Synopsis have that Reflex does not as far as C++ reflection is concerned ? What reflection problem are you trying to solve with whatever Synopsis can do ?
Synopsis provides a C++ parser together with source-code representations (such as parse tree, symbol table, etc.) exposed as C++ and python APIs (deployed as C++ libraries and python modules). As such it isn't exactly comparable to Reflex itself (IIUC), but instead, to the compiler frontend used by Reflex (i.e. gccxml). The only reflection that has been in active use for a couple of years is a documentation extraction pipeline (see http://synopsis.fresco.org/boost/ for example), though I'm working on more fine-grained control that enables building source-to-source translators, for example. Caveat: The parser isn't complete yet, as it doesn't handle template instantiation yet (for example). It is therefor not ready to serve as a drop-in replacement for gccxml. Regards, Stefan

On 5 Feb 2006, at 03:59, Stefan Seefeld wrote:
Edward Diener wrote:
What does Synopsis have that Reflex does not as far as C++ reflection is concerned ? What reflection problem are you trying to solve with whatever Synopsis can do ?
Synopsis provides a C++ parser together with source-code representations (such as parse tree, symbol table, etc.) exposed as C++ and python APIs (deployed as C++ libraries and python modules). As such it isn't exactly comparable to Reflex itself (IIUC), but instead, to the compiler frontend used by Reflex (i.e. gccxml).
I agree with you. The difference between synopsis and gccxml is the place where you intercept and retrieve the information.
The only reflection that has been in active use for a couple of years is a documentation extraction pipeline
Please note, the discussion about reflection so far was only about introspection of C++, but I think reflection is at least three different parts, the other two being - Interaction, to interact with the introspected information (e.g. construct a type, invoke a function, get/set data members) from the meta level. - Modification, to alter the introspected information at runtime (e.g. add a function member dynamically, eg. which is defined in python) -- Stefan Roiser CERN, PH Department CH - 1211 Geneva 23 Mob:+41 76 487 5334 Tel:+41 22 767 4838 Fax:+41 22 767 9425

Sebastian Redl <sebastian.redl@getdesigned.at> writes:
David Abrahams wrote:
That might be easier than you think. There's always
system("c++ whatever.cpp...")
Still unportable. The MS compiler is called cl, the Borland one bcc.
Of course, of course. That's what configuration files are for. Can't you step back and see the bigger picture?
What about different versions of GCC that might be called, and their varying ABIs?
What about them?
How to know the exact object file format produced?
Huh? Who cares?
What if the application doesn't have write access to a temporary directory to place source and object file in? For that matter, how does the application find out the temporary directory?
?? I was talking about doing code generation. You're not going to get far with generated code unless you can store it in a file. You make it sound like these are impossible programming problems, but people solve them every day.
Yes, it's possible to invoke an installed compiler to produce object code, but it creates a lot of problems on its own.
It causes no great problems. There are systems that do it with great success. -- Dave Abrahams Boost Consulting www.boost-consulting.com

David Abrahams wrote:
Of course, of course. That's what configuration files are for. Can't you step back and see the bigger picture?
I can see the bigger picture. I can certainly see the use in run-time template instantiation. I'm just speculating about the costs. I think that you underestimate them. In no way do I mean to imply that they are unsolvable. Sebastian Redl

Sebastian Redl <sebastian.redl@getdesigned.at> writes:
David Abrahams wrote:
Of course, of course. That's what configuration files are for. Can't you step back and see the bigger picture?
I can see the bigger picture. I can certainly see the use in run-time template instantiation.
I'm just speculating about the costs. I think that you underestimate them. In no way do I mean to imply that they are unsolvable.
Okay. Well consider that it's just a matter of integrating a build system and building dynamically-loaded libraries. It's been done before, and it's something we know how to do. -- Dave Abrahams Boost Consulting www.boost-consulting.com

David Abrahams wrote:
Sebastian Redl <sebastian.redl@getdesigned.at> writes:
David Abrahams wrote:
Of course, of course. That's what configuration files are for. Can't you step back and see the bigger picture?
I can see the bigger picture. I can certainly see the use in run-time template instantiation.
I'm just speculating about the costs. I think that you underestimate them. In no way do I mean to imply that they are unsolvable.
Okay. Well consider that it's just a matter of integrating a build system and building dynamically-loaded libraries. It's been done before, and it's something we know how to do.
Can you give an example of the compile-time use of reflection for templates and how this would be done ? Or have I completely misunderstood what you and Sebastian Redl are discussing ?

Edward Diener wrote:
Can you give an example of the compile-time use of reflection for templates and how this would be done ? Or have I completely misunderstood what you and Sebastian Redl are discussing ?
Although this came up in a different branch of discussion, the topic of David and my discussion was about the hardships of implementing a solution that could, at run-time, instantiate a template and load the newly generated code into the application. But for an example of compile-time reflection, I posted something a few days ago, in the thread about a policy selector using a mpl vector of policy classes. There, I used the mpl utility macro BOOST_HAS_NAMED_TYPEDEF (I think is the name of the macro.) This is one very limited example of compile-time reflection. Real language support would allow for far more interesting things here. My example implemented this compile-time pseudo-code: metafunction type policy_type(type policy) { if(policy.has_typedef("policy_type")) { return policy.policy_type; } else { return not_a_policy; } } Overloads (= specializations) could be provided for types that had a policy_type but no nested typedef policy_type. The has_typedef would basically be a part of compile-time reflection. However, reflection would be far more powerful. Instead of just testing for the existence, you could for example enumerate all typedefs. Sebastian Redl

Sebastian Redl wrote:
Edward Diener wrote:
Can you give an example of the compile-time use of reflection for templates and how this would be done ? Or have I completely misunderstood what you and Sebastian Redl are discussing ?
Although this came up in a different branch of discussion, the topic of David and my discussion was about the hardships of implementing a solution that could, at run-time, instantiate a template and load the newly generated code into the application.
Clearly one can normally, from a template definition, instantiate a template at run-time in C++ now. How is this different from what you are suggesting above ?
But for an example of compile-time reflection, I posted something a few days ago, in the thread about a policy selector using a mpl vector of policy classes. There, I used the mpl utility macro BOOST_HAS_NAMED_TYPEDEF (I think is the name of the macro.) This is one very limited example of compile-time reflection. Real language support would allow for far more interesting things here. My example implemented this compile-time pseudo-code:
metafunction type policy_type(type policy) { if(policy.has_typedef("policy_type")) { return policy.policy_type; } else { return not_a_policy; } }
OK, I understand this. In a run-time reflection mechanism, one would be able to find about type 'policy' and any nested types, member functions, or data members it contains but admittedly whether it had a nested typedef would be lost to a system at run-time.
Overloads (= specializations) could be provided for types that had a policy_type but no nested typedef policy_type. The has_typedef would basically be a part of compile-time reflection.
However, reflection would be far more powerful. Instead of just testing for the existence, you could for example enumerate all typedefs.
Agreed. There are situations where a run-time reflection mechanism would not give you what you need to know for template programming. As I understand it the solutions to this problem would be either a compile-time reflection mechanism as part of C++ or a compiler implementation, or a step which manipulates C++ source code based on compile-time reflection data before the compiler processes it. For the purposes of Boost, because it can not change every compiler on the market, and C++ has no built-in compile-time reflection, the second solution would appear ro be the only one.

On 6 Feb 2006, at 00:36, Edward Diener wrote:
Sebastian Redl wrote:
Edward Diener wrote:
Can you give an example of the compile-time use of reflection for templates and how this would be done ? Or have I completely misunderstood what you and Sebastian Redl are discussing ?
Although this came up in a different branch of discussion, the topic of David and my discussion was about the hardships of implementing a solution that could, at run-time, instantiate a template and load the newly generated code into the application.
Clearly one can normally, from a template definition, instantiate a template at run-time in C++ now. How is this different from what you are suggesting above ?
We do have a user request to do exactly this with Reflex. I think it should be possible along the ideas of this thread. What we want to do is what you were discussing, - generate the dictionary sources (with python/genreflex.py - invoking gccxml) - compile the so generated dictionary source into a module (shared lib) - dynamically open the library and load the dictionary information There is a project already at CERN which does exactly this, so it should be feasible. -- Stefan Roiser CERN, PH Department CH - 1211 Geneva 23 Mob:+41 76 487 5334 Tel:+41 22 767 4838 Fax:+41 22 767 9425

David Abrahams wrote:
Edward Diener <eddielee@tropicsoft.com> writes:
I think it is a commendable goal in Reflex to be able to reflect the parameter types for any given template instantiation, since this information may be valuable and usable at run-time. After all, at run-time, information about a template type should be just as accesible regarding finding type information than any other type. One of the pieces of information about a template type is the parameter types which are used to instantiate the template, so run-time reflection should include such information. But I feel that information about the template declaration/definition itself belongs in the realm of a compile-time reflection system, not a run-time reflection system, and can serve no purpose in a run-time reflection system.
Even if the run-time reflection system is generating code? The boundaries are a lot more fluid than you probably imagine.
I do not doubt that a compile-time reflection mechanism would be invaluable to template programming at compile time, but I do not see how such a reflection mechanism can work there unless it were integrated into the language/compiler itself since the benefits of a compile-time reflection mechanism would be to manipulate template code at compile time, similar to the way MPL does but with more built-in langauge functionality. I do not say that such an integration is not a worthy project, and to be considered as an addition to the C++ language, but I would rather separate compile-time reflection and run-time reflection since the latter can work without changing the language/compiler, as Reflex shows, while the former appears to me to need such a change. So I think it would be worthy to approach the easier task first, under the aegis of run-time functionality rather than the harder, and not mix the two in an initial attempt to provide C++ reflection. At the same time, regarding run-time reflection, and given the fact that Reflex can reflect directly from a type or an object, I would still love to see, if C++ adapts such an approach to run-time reflection, an equivalent to the built-in "typeid" operator, eg. maybe "reflection_typeid" or maybe "extended_typeid" or maybe just "typeid" generating a full reflection type ala Reflex, as part of the C++ language. But barring that, using the specific classes of Reflex at run-time works also. I just think the built-in operator way of doing it would be more elegant and more in keeping with the "typeid" idea, and the feeling that run-time reflection is part of the language.
participants (7)
-
David Abrahams
-
Edward Diener
-
Ion GaztaƱaga
-
pongba
-
Sebastian Redl
-
Stefan Roiser
-
Stefan Seefeld