Greetings, dear developers. I have some simple idea to write platform independed library to provide users with ability to load DLL/SO files (i.e. to develop plugin system etc.). Is there any library in boost? I've found nothing. Maybe we need it? Let's discuss, I am interested in your comments. Thank you very much in advance.
Vladislav Lazarenko wrote:
Greetings, dear developers.
I have some simple idea to write platform independed library to provide users with ability to load DLL/SO files (i.e. to develop plugin system etc.). Is there any library in boost? I've found nothing. Maybe we need it? Let's discuss, I am interested in your comments. Thank you very much in advance.
What functionality would such a library provide ? How would it protect users against ABI incompatibilities ? Would it include a 'type library' ? Regards, Stefan
I just want to develop wrapper around LoadLibrary, FreeLibrary, GetProcAddress for Windows and dlopen, dlclose, dlsym for Unix OSs. I think it is very easy to do and no additional functionality shall be added (maybe in another library). I tryed to design KISS (Keep It Stupid Simply) and platform-independed wrapper. -- Vladislav Stefan Seefeld wrote:
Vladislav Lazarenko wrote:
Greetings, dear developers.
I have some simple idea to write platform independed library to provide users with ability to load DLL/SO files (i.e. to develop plugin system etc.). Is there any library in boost? I've found nothing. Maybe we need it? Let's discuss, I am interested in your comments. Thank you very much in advance.
What functionality would such a library provide ? How would it protect users against ABI incompatibilities ? Would it include a 'type library' ?
Regards, Stefan _______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
I just want to develop wrapper around LoadLibrary, FreeLibrary, GetProcAddress for Windows and dlopen, dlclose, dlsym for Unix OSs.
I think it is very easy to do and no additional functionality shall be added (maybe in another library). I tryed to design KISS (Keep It Stupid Simply) and platform-independed wrapper.
-- Vladislav
For me, it seems that the library should also handle calling functions that are exported from the library that is loaded. Can this be done in a way that does not depends on the platform as the calling convention is probably different. And probably, it would be usefull to be able to remember the address of a function after we get it once and have the ability to keep the library loaded. Then when the library is unloaded? Under Windows, we typically need to unload libraries before exiting main if some libraries also load other ones and a library or the application has static object that would cause code to be executed when destroyed... If we let the system unload the dynamically loaded libraries, the system does not always knows "dynamic" depedencies and might unload the DLL in the wrong order. Philippe
Philippe Mori wrote:
I just want to develop wrapper around LoadLibrary, FreeLibrary, GetProcAddress for Windows and dlopen, dlclose, dlsym for Unix OSs.
I think it is very easy to do and no additional functionality shall be added (maybe in another library). I tryed to design KISS (Keep It Stupid Simply)
KISS stands for "Keep it simple, stupid".
Philippe Mori wrote:
I just want to develop wrapper around LoadLibrary, FreeLibrary, GetProcAddress for Windows and dlopen, dlclose, dlsym for Unix OSs.
I think it is very easy to do and no additional functionality shall be added (maybe in another library). I tryed to design KISS (Keep It Stupid Simply)
Sorry, I replied to the wrong message in the thread, and the reply was unnecessary. Please ignore.
Pavel Vozenilek wrote:
"Vladislav Lazarenko" wrote:
I just want to develop wrapper around LoadLibrary, FreeLibrary, GetProcAddress for Windows and dlopen, dlclose, dlsym for Unix OSs.
Exported symbol enumeration would be handy too.
That's all fine. The trouble really starts when you want to make assertions about the symbols as referring to C++ types / objects. If you don't, I don't really see how this topic relates to boost, or C++ in general. Regards, Stefan
Stefan Seefeld <seefeld@sympatico.ca> writes:
Pavel Vozenilek wrote:
"Vladislav Lazarenko" wrote:
I just want to develop wrapper around LoadLibrary, FreeLibrary, GetProcAddress for Windows and dlopen, dlclose, dlsym for Unix OSs.
Exported symbol enumeration would be handy too.
That's all fine. The trouble really starts when you want to make assertions about the symbols as referring to C++ types / objects. If you don't, I don't really see how this topic relates to boost, or C++ in general.
You can do some nice things with C++ interfaces: entry_point hello(some_lib, "greet"); hello(world); If you're enumerating things, you can present STL iterators. Stuff like that makes it C++-related. -- Dave Abrahams Boost Consulting www.boost-consulting.com
David Abrahams wrote:
You can do some nice things with C++ interfaces:
entry_point hello(some_lib, "greet"); hello(world);
What would that call to 'hello' do and how would you protect your application against ABI incompatibilities without heavy machinery ?
If you're enumerating things, you can present STL iterators. Stuff like that makes it C++-related.
Well, you can use C++ to enumerate symbols, but the symbols aren't (yet) related to C++, or are they ? Regards, Stefan
Stefan Seefeld wrote:
David Abrahams wrote:
You can do some nice things with C++ interfaces:
entry_point hello(some_lib, "greet"); hello(world);
What would that call to 'hello' do
Call "greet", I assume.
and how would you protect your application against ABI incompatibilities without heavy machinery ?
entry_point will probably have to demangle the various "greet" exports and perform overload resolution and implicit conversions as necessary. This _is_ heavy machinery but it's hidden from the user and Boost.Python may already contain the bulk of the functionality.
Peter Dimov wrote:
entry_point will probably have to demangle the various "greet" exports and perform overload resolution and implicit conversions as necessary. This _is_ heavy machinery but it's hidden from the user and Boost.Python may already contain the bulk of the functionality.
If this is going to become the basis for a plugin system you probably want to have some versioning mechanism as well as some form of introspection, checksumming, etc. to catch the most common failures. In short, you will really end up developing a full-fledged component model. I'm not saying this is worthless. I'm just pointing out that a simple dlopen/dlsym mechanism won't be really useful without the rest, and it's this rest that will be hard to do, in particular if the goal is to provide a generic solution. Regards, Stefan
Stefan Seefeld wrote:
Peter Dimov wrote:
entry_point will probably have to demangle the various "greet" exports and perform overload resolution and implicit conversions as necessary. This _is_ heavy machinery but it's hidden from the user and Boost.Python may already contain the bulk of the functionality.
If this is going to become the basis for a plugin system you probably want to have some versioning mechanism
Yea, which can be as simple as file containing: PluginName=foo Library=foo.so ABIVersion=3 I would like to point out that the similar versioning problem exists with boost::serialization -- if you change class definitions without changing class version, you run in problems. This however, does not render serialization completely unusable.
as well as some form of introspection, checksumming, etc. to catch the most common failures. In short, you will really end up developing a full-fledged component model.
Why do you need introspection or checksumming (what's "checksumming", btw)?
I'm not saying this is worthless. I'm just pointing out that a simple dlopen/dlsym mechanism won't be really useful without the rest, and it's this rest that will be hard to do, in particular if the goal is to provide a generic solution.
BTW, in the case where plugins target a specific version of main application, and versioning can be handled by other means (like package management), you don't have any versioning problem at all to solve in the library. - Volodya
Vladimir Prus wrote:
Why do you need introspection or checksumming (what's "checksumming", btw)?
I was thinking of some 'externalized vtable' or somesuch that would enable users of the library to modify the class' interface without breaking the application / plugin interaction. Something that initializes a proxy at plugin load time to point to the right function. Again, this may seem overkill. However, if you really know what's in the plugin you may as well load the library manually and cast the 'void *' to whatever you know the symbol is really pointing to. But then this doesn't offer anything which makes it worth including into boost IMO. Regards, Stefan
Stefan Seefeld wrote:
Vladimir Prus wrote:
Why do you need introspection or checksumming (what's "checksumming", btw)?
I was thinking of some 'externalized vtable' or somesuch that would enable users of the library to modify the class' interface without breaking the application / plugin interaction. Something that initializes a proxy at plugin load time to point to the right function.
Again, this may seem overkill. However, if you really know what's in the plugin you may as well load the library manually and cast the 'void *' to whatever you know the symbol is really pointing to.
You might know only base type of plugin and textual name, but don't know the exact type you want to create. You also don't want to invoke constructor by mangled name ;-)
But then this doesn't offer anything which makes it worth including into boost IMO.
I guess I just need to finish my library and then we can see. - Volodya
Vladimir Prus wrote:
You might know only base type of plugin and textual name, but don't know the exact type you want to create. You also don't want to invoke constructor by mangled name ;-)
That's what factories are for. In fact, that's how all C++ plugins I'v worked with so far were designed. A single entry point in terms of a C linkage function that is *assumed* to return an object of a given type (interface), and let that do all the rest. Regards, Stefan
Stefan Seefeld wrote:
Vladimir Prus wrote:
You might know only base type of plugin and textual name, but don't know the exact type you want to create. You also don't want to invoke constructor by mangled name ;-)
That's what factories are for. In fact, that's how all C++ plugins I'v worked with so far were designed. A single entry point in terms of a C linkage function that is *assumed* to return an object of a given type (interface), and let that do all the rest.
Right, and that approach is fine. But a library can add some details: 1. You won't have to write boilerplate code that creates a map from string to function that creates a class, and the entry point that lookups in the map and calls the function. 2. You will be able to pass arguments to constructor. 3. No portability problems. - Volodya
Vladimir Prus wrote:
Stefan Seefeld wrote:
Vladimir Prus wrote:
Why do you need introspection or checksumming (what's "checksumming", btw)?
I was thinking of some 'externalized vtable' or somesuch that would enable users of the library to modify the class' interface without breaking the application / plugin interaction. Something that initializes a proxy at plugin load time to point to the right function.
Again, this may seem overkill. However, if you really know what's in the plugin you may as well load the library manually and cast the 'void *' to whatever you know the symbol is really pointing to.
You might know only base type of plugin and textual name, but don't know the exact type you want to create. You also don't want to invoke constructor by mangled name ;-)
But then this doesn't offer anything which makes it worth including into boost IMO.
I guess I just need to finish my library and then we can see.
- Volodya
So, Volodya.. will you create some initial library? We can discuss it after you finish and make some additions, fixes etc. :)
Vladislav Lazarenko wrote:
But then this doesn't offer anything which makes it worth including into boost IMO.
I guess I just need to finish my library and then we can see.
- Volodya
So, Volodya.. will you create some initial library? We can discuss it after you finish and make some additions, fixes etc. :)
I'll try to do something in a couple of weeks. Basically, I only need to write docs and handle some lifetime management issue, and that will make the library usable on Unix. I think porting to Windows should not be much of a problem. Did you already look at http://cvs.sourceforge.net/viewcvs.py/boost-sandbox/boost-sandbox/libs/plugi... which is an example of intended usage. There's also implementation in: http://cvs.sourceforge.net/viewcvs.py/boost-sandbox/boost-sandbox/boost/plug... But it's undercommented ;-) - Volodya
Vladimir Prus wrote:
I'll try to do something in a couple of weeks. Basically, I only need to write docs and handle some lifetime management issue, and that will make the library usable on Unix. I think porting to Windows should not be much of a problem.
Did you already look at http://cvs.sourceforge.net/viewcvs.py/boost-sandbox/boost-sand box/libs/plugin/example/ which is an example of intended usage.
There's also implementation in: http://cvs.sourceforge.net/viewcvs.py/boost-sandbox/boost-sand box/boost/plugin/
But it's undercommented ;-)
I'd really like to collaborate with you esp. on the Windows part, since I'm very interested in such a library in one of my current projects. Regards Hartmut
On 4/29/05, Hartmut Kaiser <HartmutKaiser@t-online.de> wrote:
Vladimir Prus wrote:
I'll try to do something in a couple of weeks. Basically, I only need to write docs and handle some lifetime management issue, and that will make the library usable on Unix. I think porting to Windows should not be much of a problem.
Did you already look at http://cvs.sourceforge.net/viewcvs.py/boost-sandbox/boost-sand box/libs/plugin/example/ which is an example of intended usage.
There's also implementation in: http://cvs.sourceforge.net/viewcvs.py/boost-sandbox/boost-sand box/boost/plugin/
But it's undercommented ;-)
I'd really like to collaborate with you esp. on the Windows part, since I'm very interested in such a library in one of my current projects.
Regards Hartmut
I haven't worked on it much in ages, but http://somelib.sf.net is the dll framework I helped rewrite and extend to the platforms I needed.
Hi Hartmut,
Did you already look at http://cvs.sourceforge.net/viewcvs.py/boost-sandbox/boost-sand box/libs/plugin/example/ which is an example of intended usage.
There's also implementation in: http://cvs.sourceforge.net/viewcvs.py/boost-sandbox/boost-sand box/boost/plugin/
But it's undercommented ;-)
I'd really like to collaborate with you esp. on the Windows part, since I'm very interested in such a library in one of my current projects.
That's great! How can we arrange this collaboration? Maybe I can write quick docs describing the interface I have in mind and we can then decide if it's OK or not. And we can also look at existing libraries (which were mentioned in this thread). - Volodya
Stefan Seefeld wrote:
David Abrahams wrote:
You can do some nice things with C++ interfaces:
entry_point hello(some_lib, "greet"); hello(world);
What would that call to 'hello' do and how would you protect your application against ABI incompatibilities without heavy machinery ?
I believe that most often DLL usage is on larger scale than calling individual functions -- that is loading large classes what implement certain interface. In that case, ABI incompatibilities can be handled by some external mechanism. For example, I think in KDE there's a mechanism to describe the version of plugin, and when loading a plugin you specify the desired version. Generally handling ABI incompatibilities would require that you load debug info from the DLL, load debug info from the application, and compare declarations of all interesting classes. Certainly doable, but looks like overengineering. - Volodya
Vladimir Prus wrote:
I believe that most often DLL usage is on larger scale than calling individual functions -- that is loading large classes what implement certain interface. In that case, ABI incompatibilities can be handled by some external mechanism. For example, I think in KDE there's a mechanism to describe the version of plugin, and when loading a plugin you specify the desired version.
Sure. But what does this mean for a 'boost dso loader' library ? Would it be useful without the above ? Would it be safe ? How would you extract (C++) objects from the DSO without some type library to minimize the risk of misinterpreting symbols ? Regards, Stefan
Stefan Seefeld wrote:
David Abrahams wrote:
You can do some nice things with C++ interfaces:
entry_point hello(some_lib, "greet"); hello(world);
What would that call to 'hello' do and how would you protect your application against ABI incompatibilities without heavy machinery ?
Normally it is the programmer's responsibility to ensure that the ABI is met. In C++ standard Windows DLLs many compilers provide #pragmas for setting the ABI so that when one #includes the appropriate header file for a class or a function in a DLL, the correct ABI is specified. While a DLL/SO loading library might try to handle ABI in some way I just wanted to point out that compiler specific mechanisms often support it.
Vladislav Lazarenko wrote:
Greetings, dear developers.
I have some simple idea to write platform independed library to provide users with ability to load DLL/SO files (i.e. to develop plugin system etc.). Is there any library in boost? I've found nothing. Maybe we need it? Let's discuss, I am interested in your comments. Thank you very much in advance.
Doesn't seem very boost-ish. I don't think everything that would be a useful c++ library belongs in boost.
"Neal Becker" <ndbecker2@gmail.com> wrote in message news:d4m2lg$v21$1@sea.gmane.org...
Vladislav Lazarenko wrote:
Greetings, dear developers.
I have some simple idea to write platform independed library to provide users with ability to load DLL/SO files (i.e. to develop plugin system etc.). Is there any library in boost? I've found nothing. Maybe we need it? Let's discuss, I am interested in your comments. Thank you very much in advance.
Doesn't seem very boost-ish. I don't think everything that would be a useful c++ library belongs in boost.
I disagree - I think this library would be great for Boost, assuming the technical issues can be worked out. The C++ committee has also indicated an interest in such a library. Particularly if it can somehow paper over difference like some systems exporting all symbols by default, and others exporting no symbols by default, and requires no changes in the core language or linkers and loaders. --Beman
Beman Dawes wrote:
"Neal Becker" <ndbecker2@gmail.com> wrote in message news:d4m2lg$v21$1@sea.gmane.org...
Vladislav Lazarenko wrote:
Greetings, dear developers.
I have some simple idea to write platform independed library to provide users with ability to load DLL/SO files (i.e. to develop plugin system etc.). Is there any library in boost? I've found nothing. Maybe we need it? Let's discuss, I am interested in your comments. Thank you very much in advance.
Doesn't seem very boost-ish. I don't think everything that would be a useful c++ library belongs in boost.
I disagree - I think this library would be great for Boost, assuming the technical issues can be worked out.
The C++ committee has also indicated an interest in such a library. Particularly if it can somehow paper over difference like some systems exporting all symbols by default, and others exporting no symbols by default,
What do you mean? If a shared does not export any symbols, there's nothing you can do other than rebuilding it or modifying it to export something. Or you have portable __declspec(dllexport)? - Volodya
Vladislav Lazarenko wrote:
Greetings, dear developers.
I have some simple idea to write platform independed library to provide users with ability to load DLL/SO files (i.e. to develop plugin system etc.). Is there any library in boost? I've found nothing. Maybe we need it? Let's discuss, I am interested in your comments. Thank you very much in advance.
I have some initial work in boost sandbox, directory libs/plugin. However, I'm more interested in higher-level functionality -- given a shared library create an instance of class defined there, given name of the class and a set of constructor arguments. - Volodya
Vladimir Prus wrote:
I have some initial work in boost sandbox, directory libs/plugin. However, I'm more interested in higher-level functionality -- given a shared library create an instance of class defined there, given name of the class and a set of constructor arguments.
- Volodya
Hm.. it will aplly some dependensies to shared library development. I mean it is impossible for example to get a set of constructor argument of the class stored in the shared library without shared library development agreement. So, if we want to go this way then we need to define some base interface for the class and all of the classes in SO/DLL should implement it. Moreone, for class X we should have some description struct which will have information about that class (at least class type), a pointer to the function which will create an instance of the class, information about arguments (possible with its description), pointer to the function which creates an instance of that class (Class *createInstance(...)). And also a list of exported classes (extern "C" struct with a list of description structs), init and fini methods (to hide DLLMain/_init/_fini from the user). Am I right?
Vladislav Lazarenko wrote:
Vladimir Prus wrote:
I have some initial work in boost sandbox, directory libs/plugin. However, I'm more interested in higher-level functionality -- given a shared library create an instance of class defined there, given name of the class and a set of constructor arguments.
- Volodya
Hm.. it will aplly some dependensies to shared library development. I mean it is impossible for example to get a set of constructor argument of the class stored in the shared library without shared library development agreement. So, if we want to go this way then we need to define some base interface for the class and all of the classes in SO/DLL should implement it.
Sure. I doubt there are many use case where you can use a symbol in a DLL knowing only its name ;-)
Moreone, for class X we should have some description struct which will have information about that class (at least class type), a pointer to the function which will create an instance of the class, information about arguments (possible with its description), pointer to the function which creates an instance of that class (Class *createInstance(...)). And also a list of exported classes (extern "C" struct with a list of description structs), init and fini methods (to hide DLLMain/_init/_fini from the user).
Am I right?
Yes. In my design shared library keeps a map<string, any>. When you want to create a plugin, you specify a name and a base type. That map is accessed for and the 'any' value, if found, is cast to 'abstract_factory<BasePluginType>' and then a 'create' method of 'abstract_factory' is called. For each base plugin type, you specialize a template class called 'virtual_constructors', which specifies all constructor signatures you may use when creating plugins of those types. So, abstract_factory<BasePluginType> will have one 'create' overload for each signature. The information about arguments is not represented at run-time, as I can get everything to work with static type information. - Volodya
On Tue, 2005-04-26 at 19:01 +0300, Vladislav Lazarenko wrote:
Greetings, dear developers.
I have some simple idea to write platform independed library to provide users with ability to load DLL/SO files (i.e. to develop plugin system etc.). Is there any library in boost? I've found nothing. Maybe we need it? Let's discuss, I am interested in your comments. Thank you very much in advance.
Some existing practice: http://www.s11n.net/class_loader/ Looks like it needs some work to be Boostified, but the author has released it into the public domain. -Jonathan
participants (14)
-
Beman Dawes -
David Abrahams -
Edward Diener -
Hartmut Kaiser -
Jonathan Brandmeyer -
Neal Becker -
Pavel Vozenilek -
Peter Dimov -
Philippe Mori -
Russell Hind -
Stefan Seefeld -
Thomas Matelich -
Vladimir Prus -
Vladislav Lazarenko