Hi Janek! I can answer a few of these:

1. do you plan to use (not yet accepted) Boost.UUID to uniquely identify classes for the class factory?

The library allows you to define an arbitrary type identification facility. Check out the typeinfo header. It uses RTTI names by default - but you can identify your constructors by version number or any other mechanism. I'd say that Boost.UUID would be a good fit. We'll try to add an automatic way of using it (but not require it). For instance, the factory_map class is actually just a typedef: typedef basic_factory_map<default_type_info> factory_map; You could instead define your own, by following the conventions in the typeinfo header. This will be detailed in an example. Then you could do: typedef basic_factory_map<boost_uuid_type_info> uuid_factory_map; And use it just like a factory_map.

2. how else do you identify them if not with UUID ?

RTTI. Some platforms (OS X) actually work fine just using the type_info reference returned by typeid - other platforms (Windows, for instance) only work with the full typeid(x).name(). But see above for other options.

3. I remember that Boost.Serialization is also using its own home-brewn GUID called extended_type_info. It would be really nice if both libraries (serizalization and extension) used the incoming UUID library, instead of some home-brewn solutions. Any comments on that?

I agree. We ought to standardize it. For now though, we'll probably include a header to use extended_type_info, a header to use uuid, and leave the default as RTTI.

4. Do you plan some integration with serialization library? (serialization after all, needs to create classes during the deserialziation process, it's a crucial part of this process, in fact).

I, personally, hope so. But it is not planned for SoC currently. If someone would like to volunteer ...

5. does your library work without bjam? (I remember that Jeremy Pack configured Jamfile to use *.extension). I'm particularly concerned with that, because I don't want to change the build system in my project ;)

Yep - it works fine without bjam. The basic functionality currently works without Boost at all (though advanced functionality requires it - as do the tests). You don't have to rename the files. You can just use *.so, *.dll or whatever. I just used a different extension for ease of cross-platform use.

6. do you have a simple example, with a different build system? I'd prefer make (ie. makefile), as it is simplest to adapt to any other build system that I know.

I believe it should be quite straightforward. Just put the boost/extension directory in the system include path. Then use it in your project. You can use the functionality you'll need with only the STL. Only some of the "convenience" functions depend on Boost. If you don't include those headers, you won't need Boost installed (though why you wouldn't want to use the rest of Boost if you were already using this library...).

7. can you explain shortly what Reflection is? Or some URL with explanation, please.

I think of reflection as runtime type information about the methods of a class for which you do not have access to any of its base classes. As such, there are a lot of directions one could go with it - I'm not sure which direction Mariano will choose. Some of these directions are really hard in C++. I think we'll be discussing that a lot in about three weeks - and will want lots of input from the community.

8. You need to record information about classes stored in plugins. Does this information include also the inheritance hierary? I mean - do you know on runtime which class derives from which one?

Yes and no. When you export a class as a plugin, you must declare which interface it is being exported as. As such, you must have a different declaration for each interface it is being exported as: current_factory_map.add<flying_car, vehicle, std::string>("A flying car exported as a vehicle"); current_factory_map.add<flying_car, car, std::string>("A flying car exported as a car"); Here, a flying car is exported first through its vehicle interface, then through its car interface. If we wanted to allow just exporting a flying_car and let it be loaded through any of it's interfaces, a whole bunch of dynamic casts could do it. I don't believe this functionality is necessary. So, the only inheritance information we know is what is explicitly declared when the class is added as a plugin. You could always use the Info template type to store information about the inheritance hierarchy, if you need that information.

...

I'll send a mail like this periodically (probably once a week or two) to let you know the advances.

It would make things easier for me (and others!), if in each periodical post from you, you will give an URL (or svn command with svn:// addres) to download your library, and maybe an URL to documentation (at its current stage, of course - I absolutely don't expect to see finished documentation on start!).

The best place to get it is still the sandbox. We'll put periodic releases in there zipped. The last release is already out of date though. (I put it up a week ago, but Mariano added some new good examples). As such, I recommend pulling down the sandbox with SVN and syncing it once in a while. If you just want to browse the latest release though, check out this URL: http://svn.boost.org/trac/boost/browser/sandbox/libs/extension Jeremy On 6/6/07, Janek Kozicki <janek_listy@wp.pl> wrote:

Mariano Gabriel Consoni said: (by the date of Wed, 06 Jun 2007 14:07:36 -0300)

...

As some of you may know, I'm working on a SoC project called "Boost.extensions and reflection".

great to hear from you! :-)

I still didn't download your library to test it locally. I'll do it not later than next week, I'm sure of that.

Some questions I have:

1. do you plan to use (not yet accepted) Boost.UUID to uniquely identify classes for the class factory?

2. how else do you identify them if not with UUID ?

3. I remember that Boost.Serialization is also using its own home-brewn GUID called extended_type_info. It would be really nice if both libraries (serizalization and extension) used the incoming UUID library, instead of some home-brewn solutions. Any comments on that?

4. Do you plan some integration with serialization library? (serialization after all, needs to create classes during the deserialziation process, it's a crucial part of this process, in fact).

5. does your library work without bjam? (I remember that Jeremy Pack configured Jamfile to use *.extension). I'm particularly concerned with that, because I don't want to change the build system in my project ;)

6. do you have a simple example, with a different build system? I'd prefer make (ie. makefile), as it is simplest to adapt to any other build system that I know.

7. can you explain shortly what Reflection is? Or some URL with explanation, please.

8. You need to record information about classes stored in plugins. Does this information include also the inheritance hierary? I mean - do you know on runtime which class derives from which one?

...

I'll send a mail like this periodically (probably once a week or two) to let you know the advances.

It would make things easier for me (and others!), if in each periodical post from you, you will give an URL (or svn command with svn:// addres) to download your library, and maybe an URL to documentation (at its current stage, of course - I absolutely don't expect to see finished documentation on start!).

-- Janek Kozicki | _______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost

Johan,

Yes, please don't make it a requirement to use UUIDs. I'll definitely have a need to use human-readable type names, which could also easily map to shared library names. This would allow for some automatic loading of shared libraries based upon the requested types.

One of the primary goals of the library is to keep the separate parts as encapsulated as possible. As such, if the uuid specific header was included, they could be used. Otherwise, there would be no dependencies on uuid. You can define any arbitrary mapping for your class's type information. You could create a complicated struct to describe them, as long as you provided a way to compare them (a less than operator). So, I believe that what you propose should not be an issue. I or Mariano will be writing an example soon showing the use of different types of type info.

[snip]

...

...

2. how else do you identify them if not with UUID ?

RTTI. Some platforms (OS X) actually work fine just using the type_info reference returned by typeid - other platforms (Windows, for instance) only work with the full typeid(x).name(). But see above for other options.

Plain strings, e.g. equal to class names (including namespaces):

"my::namespace::MyClass"

I'll make sure to detail that specific example in the tutorial.

Why not provide a class loader with the library, which could be configured with e.g. :

- Locations (i.e. directories) to use for loading, in prioritized order.

- Type id => library name mappers, allowing the user to only provide the type id when loading classes.

- Shared library extensions to use for the library loading (defaults to platform standard extension).

- Explicit load functionality, where the user can specify the library name and the type id. As for cross-platform usage, the users should normally not include the file extensions or the paths to the libraries, only the canonical library names. It should still be _possible_ to provide the exact path and library extension.

Yes, this is planned - enough people have mentioned it, that we'll definitely work it in. The basic design of it is done, and it will be implemented "when we have time". Once again, it will be included in a "convenience" header, and the code for it will only be included if its header is included explicitly. Since not everyone will need this functionality, it will be made optional.

[snip]

...

...

7. can you explain shortly what Reflection is? Or some URL with explanation, please.

I think of reflection as runtime type information about the methods of a class for which you do not have access to any of its base classes. As such, there are a lot of directions one could go with it - I'm not sure which direction Mariano will choose. Some of these directions are really hard in C++. I think we'll be discussing that a lot in about three weeks - and will want lots of input from the community.

I don't know what you intend to do, but please don't mix the two libraries into one. I'd even go as far as to suggest that the "dynamic (un-)loading of libraries/finding the entry points for specific methods" part of the Extension library should probably be a standalone library.

We will be posting the proposed reflection solution to this list before implementing it. We'll do our best to avoid high coupling between the plugin loading and reflection. I thought about this for a while this morning, and I think that we can solve this in a way where the reflection will work without including any of the plugin loading headers, and vice versa. However, a "convenience" header will be provided to make it easy to use them together. We'll definitely want your input when we put forward the proposed design.

I assume that it will always be possible to dynamic_cast the created instance to whatever types it actually implements, right? Example:

boost::shared_ptr<base> pb = "load extension and create shared base instance"; boost::shared_ptr<derived> pd = boost::dynamic_pointer_cast<derived>(pb);

Almost always - you of course can't cast it to a type that you don't have defined in the current module. In addition, some compilers have trouble doing dynamic casts across shared library boundaries. I do know that an older version of the Borland C++ compiler had problems with this, but am not sure about other compilers. I have done it successfully with relatively recent versions of VS and gcc. Thanks for the suggestions. We'll work on them! Cheers, Jeremy

Johan Nilsson said: (by the date of Thu, 7 Jun 2007 08:27:11 +0200)

Jeremy,

sorry for jumping into this discussion. Just a few points/questions.

there's nothing to sorry about. Really - we are all glad that you join the sub-community interested in plugin loading. Welcome! :) -- Janek Kozicki |

We're actually planning on writing an example "C++ interpreter", partially just for debugging reflected classes. I'd actually want to separate the "reflection" of the methods from the declaration of the class - the same as Boost.Extension does not require any plugin declarations in the class header file. So if we had the following class: class Car { private: // whatever private stuff public: bool start(); bool turn(float angle); }; Then we could reflect it as: boost::extension::reflection car_reflection<Car, std::string>("A Car!"); /* Here we declare that we are reflecting out a Car. The second template parameter declares that we will describe this reflection using a string. Any arbitrary type could be used here. */ car_reflection.add<int>(&Car::start, 3); car_reflection.add<std::string, std::string>(&Car::turn, "turn", "turn_angle"); /* Here we add two methods to the reflection. The library must correctly parse and remember the parameters and return type of these methods. Here, we elect to describe the first method using a number and the second using a string. For the second method, the library knows now that it is a function of Car called turn that takes a float named "turn_angle". */ This allows us to choose exactly which methods are reflected. The techniques I used above are very similar to the methods used for declaring plugins in Boost.Extension, and should be clear to those who have examined those. Now you can export it using the Boost.Extension methods as usual. You'd use the car_reflection variable as the Info parameter, and export a factory for a generic reflected_object. Then you could create the reflected_object, initializing it with a reference to the reflected_car object, and then begin using it in whatever module you're in. Now, the real trick is how we will then call those methods. There are a number of possible ways of doing this. I think we'll implement a number of different methods and let the community decide which ones are the most useful. I'd imagine we'd want about three different ways of accessing these reflected methods: 1 - Call functions when you know the exact identifier (whatever type you decided the identifier should be ...), parameters, and return value. 2 - Call functions when you just put a whole bunch of possible parameters into a map, and then the call succeeds if it can find the values it needs, for instance: parameter_map.add<float, std::string>(.5f, "turn_value"); // A parameter named // red_value (we're describing it with a string), // which is a float with a value of .5f my_reflected_car->call<std::string>("turn", parameter_map); // Now we call the turn // function, which looks for the values it needs // in the parameter_map. If it finds them, the call // succeeds, and a value is returned (boost::any probably). Anyway, the above are just my thoughts about how I'd like to see the interface. Since Mariano is writing it, he'll have to decide. Jeremy On 6/7/07, Janek Kozicki <janek_listy@wp.pl> wrote:

Jeremy Pack said: (by the date of Wed, 6 Jun 2007 11:25:20 -0700)

...

...

7. can you explain shortly what Reflection is? Or some URL with explanation, please.

I think of reflection as runtime type information about the methods of a class for which you do not have access to any of its base classes. As such, there are a lot of directions one could go with it - I'm not sure which direction Mariano will choose. Some of these directions are really hard in C++. I think we'll be discussing that a lot in about three weeks - and will want lots of input from the community.

I see... Some time ago I've been thinking about similar thing also: I wanted to create some Scriptable interface. The C++ coder will register methods in his class using some macros. And later those methods could be called by some kind of interpreter reading from std::cin.

Eg. you could register method 'int Foo(int)', and later when the interpreter reads from std::cin a string "Foo(42)" it would parse the input and call this method with appropriate argument, then print the return value.

I see where the word 'reflection' comes from - the methods of a class are reflected in some runtime information.

I abadoned this project, but the only way that I could see is to use macros - one macro per method. Which effectively doubles the length of *.hpp file with class definition.

Maybe it is possible to write some really magic macro, which would allow to declare a method in the class and register it in the same line:

class Bar { MAGIC(int Foo(int);) };

that isn't going to work ;) what to do with brackets, etc? What's with that semicolon? ;-)

-- Janek Kozicki | _______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost

...

I'd actually want to separate the "reflection" of the methods from the declaration of the class - the same as Boost.Extension does not require any plugin declarations in the class header file.

where those declarations are supposed to be placed, then?

The user declares an extern "C" function and puts the declarations there. The reasoning behind it was: 1 - The class shouldn't have to know it will be a plugin. 2 - Only one extern "C" function is now required. 3 - It is probably no more verbose. 4 - All plugin declarations for a library are in one place (though, of course, you could declare multiple extern "C" functions with different names, and then instruct the Extension library to load each of them). 5 - It is thread safe - you could have multiple threads loading the same library at the same time, and it wouldn't cause any problems. No mutexes necessary. So, basically, you just put into some .cpp file (it could actually be your class's .cpp file) this extern "C" function. I believe it uses far less code than the other solutions that I've looked at - which should help make it easier to avoid or fix errors. My preferred method is as follows (though the following is certainly not required by the Extensions library - it is pretty flexible): 1 - Put a single extern "C" function into every shared library that declares the plugins exported by that library. 2 - Give the extern function the same name in every library. This basically just gives each library one extra function that describes all of the plugins available in it, while not requiring modification of the classes that are being exported. This should make it rather simple to fit into existing projects. Jeremy

...

From my point of view it's better to have a single place to update information about a class. When method signature changes I prefer to update just two files (.cpp & .hpp), instead of three files (.cpp & .hpp & .hpp) (not mentioning their callers). I'll certainly always forgot about the third file, which will lead to compilation errors, or crashes.

But if those .hpp files are supposed to reside in the same directory and belong to the same plugin, then the solution can make sense. eg. car.hpp and car_reflection.hpp

However the ability of connecting both files to be a single file will certainly be useful in some cases. Please don't disallow that. I'm not sure which I'll really prefer, but one file less to decrease human memory burden is very temping.

...

boost::extension::reflection car_reflection<Car, std::string>("A Car!");

side note: wouldn't it be boost::reflection car_reflection<...> since as I understand, the reflection will come out as a separate library?

A very annoying problem I encountered when implementing my own plugin and serialization library was that templates sometimes are not handled correctly across plugins by the compiler. I hope that you know what I mean:

I used templates in serialization code, to determine the type of serialized object. The template function in question was called findType<>();

Some plugin code uses that template. The .hpp template code resides in a serialization plugin compiled earlier.

It can happen that sometimes the compiler is not generating a code for findType() function. It is assuming that the code for this function is in serialization library compiled earlier. Which cannot be true, because when serialization was being compiled the object type wasn't known yet. The effect is that plugin is calling a fallback findType() function, that gives an error "cannot determine the type!".

Have you had this problem?

How do you plan to cope with a problem of templates across plugins? It certainly exists, and honestly I don't know where it comes from, and how to cope with it.

I'm asking about that, becuase your design decision is to put all Reflection information inside templates, like the car_reflection<> above.

-- Janek Kozicki | _______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost