[extension] Suggestion: Loading shared libraries from memory
Hi, I'm not a huge expert on shared libraries, however I was wondering if there's a portable way to support loading libraries from a memory buffer instead of from the file system. If this is possible, could it be added as a feature to Boost.Extension? Thanks for reading!
Robert, Could you be a little bit more specific as to what you'd like? What is the exact use case? When a shared library is loaded, it is shared by all processes using it. It is also unloaded in the way memory can be handled by a garbage collecter - some time after all references to it have been removed. As such, the first process to load it is, in effect, buffering it into memory, and other processes can then load it very cheaply. As far as loading it directly from a memory buffer, that would require copying a bunch of routines from the platform-specific functions that fix pointer references, function references, type information etc. within a shared library when it is loaded. If you're sending shared libraries over a network connection between computers, I can see how this might possibly be more optimal. But if you're shooting to make your program that heavily optimized, I'd recommend assembly language :) So, yes, it is possible. No, I don't think it is worth attempting. Jeremy On Fri, Jul 11, 2008 at 9:39 AM, Robert Dailey <rcdailey@gmail.com> wrote:
Hi,
I'm not a huge expert on shared libraries, however I was wondering if there's a portable way to support loading libraries from a memory buffer instead of from the file system. If this is possible, could it be added as a feature to Boost.Extension?
Thanks for reading! _______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
On Fri, Jul 11, 2008 at 12:02 PM, Jeremy Pack <rostovpack@gmail.com> wrote:
Robert, Could you be a little bit more specific as to what you'd like? What is the exact use case?
When a shared library is loaded, it is shared by all processes using it. It is also unloaded in the way memory can be handled by a garbage collecter - some time after all references to it have been removed.
As such, the first process to load it is, in effect, buffering it into memory, and other processes can then load it very cheaply.
As far as loading it directly from a memory buffer, that would require copying a bunch of routines from the platform-specific functions that fix pointer references, function references, type information etc. within a shared library when it is loaded.
If you're sending shared libraries over a network connection between computers, I can see how this might possibly be more optimal. But if you're shooting to make your program that heavily optimized, I'd recommend assembly language :)
So, yes, it is possible. No, I don't think it is worth attempting.
Thanks for responding Jeremy. My use case was simple. I would receive a DLL in memory over a network, and instead of first writing it to a file and THEN opening it, I was hoping I could just read it from memory as if it were a file. However, I've found out that I no longer need to do this. I'll first be writing it to a file, so this feature would no longer be needed. Thanks for emphasizing a few good points about this, it's probably best to keep it on the file system.
You may want to take a look at ATL (the ActiveX Template Library) and CORBA Components. The trouble with loading DLL over a network are many: need to worry about security, about correctness (distributing a DLL that works for most, but breaks one, and you need to "unload" them all), about DLL dependencies (*NIX and Windows handles this differently), and about interface (header) dependencies, about memory ownership across this interface, error reporting, etc etc etc. In other words, almost every attempt to use the shared library mechanism to deploy software components eventually morphs into Java or C#. For C/C++, the best use of Shared libs and DLLs remains the original purpose: to make idenical memory images for sections of code or data that is shared by otherwise unlike processes. Indeed, the typical way to load DLL's over a network is to place the dll on a network drive. That said, a good read on the subject remains Copliens "Advanced C++" where he manages to take the .o file itself and load it. -----Original Message----- From: Robert Dailey [mailto:rcdailey@gmail.com] Sent: Monday, July 14, 2008 10:30 AM To: boost@lists.boost.org Subject: Re: [boost] [extension] Suggestion: Loading shared libraries frommemory On Fri, Jul 11, 2008 at 12:02 PM, Jeremy Pack <rostovpack@gmail.com> wrote:
Robert, Could you be a little bit more specific as to what you'd like? What is
the exact use case?
When a shared library is loaded, it is shared by all processes using it. It is also unloaded in the way memory can be handled by a garbage collecter - some time after all references to it have been removed.
As such, the first process to load it is, in effect, buffering it into
memory, and other processes can then load it very cheaply.
As far as loading it directly from a memory buffer, that would require
copying a bunch of routines from the platform-specific functions that fix pointer references, function references, type information etc. within a shared library when it is loaded.
If you're sending shared libraries over a network connection between computers, I can see how this might possibly be more optimal. But if you're shooting to make your program that heavily optimized, I'd recommend assembly language :)
So, yes, it is possible. No, I don't think it is worth attempting.
Thanks for responding Jeremy. My use case was simple. I would receive a DLL in memory over a network, and instead of first writing it to a file and THEN opening it, I was hoping I could just read it from memory as if it were a file. However, I've found out that I no longer need to do this. I'll first be writing it to a file, so this feature would no longer be needed. Thanks for emphasizing a few good points about this, it's probably best to keep it on the file system. Visit our website at http://www.ubs.com This message contains confidential information and is intended only for the individual named. If you are not the named addressee you should not disseminate, distribute or copy this e-mail. Please notify the sender immediately by e-mail if you have received this e-mail by mistake and delete this e-mail from your system. E-mails are not encrypted and cannot be guaranteed to be secure or error-free as information could be intercepted, corrupted, lost, destroyed, arrive late or incomplete, or contain viruses. The sender therefore does not accept liability for any errors or omissions in the contents of this message which arise as a result of e-mail transmission. If verification is required please request a hard-copy version. This message is provided for informational purposes and should not be construed as a solicitation or offer to buy or sell any securities or related financial instruments.
participants (3)
-
Jeremy Pack -
Lance.Diduckļ¼ ubs.com -
Robert Dailey