[Interprocess] How to prevent collision of mapping addresses with DLL Loading on Windows?
Hi, I do something like fixed_managed_shared_memory segment (open_only, "MyFixedAddressSharedMemory" //Shared memory object name ,(void*)0x30000000 //Mapping address to establish an shared memory segment which gets faded into the virtual address space of every process taking part on the IPC at the same offset (this is important for me). This works quite well for most processes. But problems arise with IPC-clients using a lot of delay loaded DLL's. This seems to be especially to be true for MFC windows programs. While normal loaded DLL's goes to 0x10000000, or their fixed address or are relocated to some there like 0x0..., some windows delay loaded DLL's seem to can be placed nearly anywhere inside the virtual address space, This makes the success of establishing the IPC into a lottery game. For example I used 0x40000000 for a shared memory range in the IPC clients. One client used windows MFC. Taking part on the IPC inside this client should happen after some pushbutton was pressed but it failed sometimes. I found out that in the moment of the button press windows delay loaded two additional DLL's which unfortunately faded in at the memory range I configured for the IPC (around 0x40000000). I was not able to find a 100% satisfying solution. The best thing I came up with was: -link everything you can static -Link the self produced DLL's with fixed base address -Allocate the memory range as the first call in main() using OS-specific functions like VirtualAlloc() -Free the allocated memory just before initializing the IPC (so it's sure that no delay loaded DLL can be faded in in the mean time) -for the rest (mainly windows or other 3rd party DLLs) hope the best This seem to work at least for now but I guess there must be a more elegant solution. I did google a lot about the MS loader/run time linker, but there seems to be no option to prevent it from selecting a certain address range. Could anybody help me out? Thank you in advance! Best Regards Joerg
pheres wrote:
-Allocate the memory range as the first call in main() using OS-specific functions like VirtualAlloc()
This is what I do. It's not 100% reliable (because at some point, you're going to have to VirtualFree the slice so the shared memory lib can use it), but it's better than most alternatives. If *you're* writing the code that uses the shared memory block, I would *strongly* encourage you to attempt to re-write it so that you don't need to have it loaded into the same address location in all processes. The only reason I researched and use the "first in main" trick is that I'm using a vendor-supplied library that really really really wants a certain location. Needless to say, I've been on their case to fix that, since it's brittle, at best, and completely unworkable in certain cases (like, where I'm writing a lib, and using their lib, but I didn't write "main").
Eric J. Holtman wrote:
It's not 100% reliable (because at some point, you're going to have to VirtualFree the slice so the shared memory lib can use it), but it's better than most alternatives. Yes, I VirtualFree exactly one statement before initializing the IPC.
If *you're* writing the code that uses the shared memory block, I would *strongly* encourage you to attempt to re-write it so that you don't need to have it loaded into the same address location in all processes. I have a tree like structure in the shared memory using pointers (which have to be valid inside every process, therefore the need for fixed base address) as connection elements. The structure is fairly big and queried right often. If I use offsets instead pointers and convert that to pointers using the difference in the base addresses I get about 1/10th of the original performance.
(like, where I'm writing a lib, and using their lib, but I didn't write "main"). You could call VirtualAlloc() inside DllMain() of your lib as a workaround.
I found one other possibility to get some code (which could alloc and therefore reserve the memory) executed possibly before any run time linking occurs: Creating a thread local storage callback. But it involves some PE format editor tools and probably manual work. Seems to hacky for anything but creating malware. I can't believe that Windows does not offer a way to solve this problem more elegantly, e.g. through setting some option somewhere which states like "if you load any DLL, do it, but do not relocate it between 0x... and 0x...". MS should know that the described problem could happen, but I'm unable to find something in their docs. Best Regards Joerg
pheres wrote:
Eric J. Holtman wrote:
It's not 100% reliable (because at some point, you're going to have to VirtualFree the slice so the shared memory lib can use it), but it's better than most alternatives. Yes, I VirtualFree exactly one statement before initializing the IPC. If your app is multithreaded, this isn't a failsafe solution.
It will work, exactly until your big demo in front of the VC guys.
right often. If I use offsets instead pointers and convert that to pointers using the difference in the base addresses I get about 1/10th of the original performance.
I understand the problem, I'm just telling you that if your requirements are "must work all the time", then something has to give, and that something is going to be performance.
You could call VirtualAlloc() inside DllMain() of your lib as a workaround.
That doesn't help me, if someone else has already loaded something there, as you've already discovered.
format editor tools and probably manual work. Seems to hacky for anything but creating malware.
Right.... this is the problem if you don't have the source for main().
I can't believe that Windows does not offer a way to solve this problem more elegantly, e.g. through setting some option somewhere which states like "if you load any DLL, do it, but do not relocate it between 0x... and 0x...".
How's that supposed to help, if you don't have the source for main? You really want your DLL to load at 0x30000000. What happens if some other vendor also wants that address space? At that point, if you're writing a library, and you haven't made yours address indifferent, the end user is screwed. He can't use both.
How's that supposed to help, if you don't have the source for main?
You really want your DLL to load at 0x30000000. What happens if some other vendor also wants that address space? At that point, if you're writing a library, and you haven't made yours address indifferent, the end user is screwed. He can't use both. I would like you stated indirectly as well probably never use that kind of IPC for shrink-wrapped software. In my project I have all the source and all the control, the end user can't touch anything but some gui which starts up if he presses the power button I deliver to him together with all other hardware. For me is the performance the critical point which forces me to use
Eric J. Holtman wrote: the fixed base address kind of shared memory. And the main problem is windows with it's (apparent?) non-configurable DLL delay load relocation address algorithm. Best Regards Joerg
pheres wrote:
I would like you stated indirectly as well probably never use that kind of IPC for shrink-wrapped software. In my project I have all the source and all the control, the end user can't touch anything but some gui which starts up if he presses the power button I deliver to him together with all other hardware.
Well, then you're probably OK.
is windows with it's (apparent?) non-configurable DLL delay load relocation address algorithm.
I'm still trying to figure out how you'd want that to work, even if you could design it... but that's another (non Boost) discussion.
Eric J. Holtman wrote:
I'm still trying to figure out how you'd want that to work, even if you could design it... but that's another (non Boost) discussion. It would of course not work if some DLL need to be loaded which can not be relocated. But they are ok anyway because I have the chance to know up front where they go inside the virtual address space. For all other DLL's it would be enough to relocate some where (thats apparently what's windows doing) but not inside the configured region. Or do I miss something?
Best Regards Joerg
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hello! Am 14.12.2009 um 15:41 schrieb pheres:
If *you're* writing the code that uses the shared memory block, I would *strongly* encourage you to attempt to re-write it so that you don't need to have it loaded into the same address location in all processes. I have a tree like structure in the shared memory using pointers (which have to be valid inside every process, therefore the need for fixed base address) as connection elements. The structure is fairly big and queried right often. If I use offsets instead pointers and convert that to pointers using the difference in the base addresses I get about 1/10th of the original performance.
You could try a self-relative offset. The offset is signed and you add its value to its address after reading it. It should not cost too much CPU. OF course, it only works if your address range is less the 2GB. 73, Mario - -- Mario Klebsch mkl@innodyne.de Innovative Dynamics GmbH http://www.innodyne.de Wiesenweg 9 Tel: +49 5304 91 19 410 38533 Vordorf / Rethen Fax: +49 5304 91 19 412 Geschäftsführer: Dipl. Phys. Gerhard Heidkamp Sitz: 38533 Vordorf - Rethen, Wiesenweg 9 Handelsregister beim Amtsgericht Hildesheim HRB 100553, USt-Id Nr. DE 215 876 442 SICHERHEITSHINWEIS: =================== Diese Email ist elektronisch signiert. InnoDyne versendet grundsätzlich seit dem 11.9.2006 verbindliche Dokumente nur als signierte PDF - Dateien oder elektronisch nachprüfbar signierte Emails. Juristisch und technisch verbindliche Dokumente dürfen das Haus InnoDyne nicht als andere Datei- formate verlassen. Sollten Sie ein signiertes, technisch oder kauf- männisch/juristisch verbindliches Dokument in einem anderen Dateiformat als PDF oder signierte Email von InnoDyne erhalten, so ist dies sehr wahrscheinlich ein Versehen des Versenders, eine Software - Fehlfunktion oder schlimmer noch, eine Fälschung. Wir bitten in einem solchen Fall um sofortige Information und Übermittlung solcher Dokumente an gl@innodyne.de -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.8 (Darwin) iEYEARECAAYFAksmW+sACgkQDwNq2neCwq4n1ACeLQGYqbihh03oAqUGF1mtpbAv dlAAniiStVfGTtEPnYXTRFIMzQOL/Isc =uxwf -----END PGP SIGNATURE-----
participants (3)
-
Eric J. Holtman -
Mario Klebsch -
pheres