[interprocess] sharing memory between 32 bit and 64 bit Windows processes
Hello Ion,
- In a mixed environment, each shared heap can be no larger than 32 bit. - offset_ptr must have the ability to point to any place _in any of potentially many heaps_ because at the time of dereferencing it has no information about the heap it is meant to point to. This requires a heap-relative 32 bit quantity plus some "heap selector", or a this-relative 64 bit quantity. The latter is simple, fast and not significantly less compact than the former, so we pick the latter. - Option 1: For the rest of the code, the easiest option is to make everything 64 bit, e.g., deriving size_type et. al. from offset_ptr. Unfortunately, this makes all interfaces 64 bit. So even on 32 bit you can request a 64 bit shared heap. Internally, when going to the OS to allocate the heap, there is a cast to 32-bit size_t with data loss, which is ugly. - Option 2: So the interfaces should really be 32 bit on both 32 bit and 64 bit. This would require a second template parameter size_type for the MemoryAlgorithm, in addition to the pointer type. We can then migrate parts of the code to use this 32 bit quantity as much as possible for compactness. I think this Option 2 is safer and philosophically closer to what is really going on.
- Option 3: Actually, the right solution is to make offset_ptr::difference_type of a mixed-environment offset pointer 32 bit, but its internal offset representation 64 bit. So all shared size_types and difference_types will be 32 bit, which is correct because all array sizes and differences between pointers within arrays are limited to 32 bit. For those places where internal algorithms (such as the rbtree memory algorithm) need modulo arithmetic, we will make a custom typedef in offset_ptr to export the "virtual address space size". Any objection? It would be great if you could give us feedback, because we want to get done with that stuff, we need it badly... Thanks! Arno -- Dr. Arno Schödl | aschoedl@think-cell.com Technical Director think-cell Software GmbH | Chausseestr. 8/E | 10115 Berlin | Germany http://www.think-cell.com | phone +49 30 666473-10 | US phone +1 800 891 8091 Amtsgericht Berlin-Charlottenburg, HRB 85229 | European Union VAT Id DE813474306 Directors: Dr. Markus Hannebauer, Dr. Arno Schoedl
El 24/02/2011 9:37, Arno Schödl escribió:
- Option 3: Actually, the right solution is to make offset_ptr::difference_type of a mixed-environment offset pointer 32 bit, but its internal offset representation 64 bit. So all shared size_types and difference_types will be 32 bit, which is correct because all array sizes and differences between pointers within arrays are limited to 32 bit. For those places where internal algorithms (such as the rbtree memory algorithm) need modulo arithmetic, we will make a custom typedef in offset_ptr to export the "virtual address space size".
Any objection? It would be great if you could give us feedback, because we want to get done with that stuff, we need it badly...
Differentiating "shared distances" from other types is easy. Pointer arithmetics/function parameters, etc. can create temporary pointers with point from stack to shared memory and this distance will be different in 32 bit (max. 32 bit distance) and 64 systems (max. 64 bit distance!). The pointer type is the same, so I'm afraid option 3 is not possible. Option 2 suffers from the same problem. You can make pointers 32 bit but again 32 bit distance from stack to shared memory might be too short in 64 bit environments. For options 1, the OS won't let you allocate/connect to memory bigger than 32 bit limits, so you will get an error when connecting/creating such shared memory. Best, Ion
Hello Ion,
Differentiating "shared distances" from other types is easy. Pointer arithmetics/function parameters, etc. can create temporary pointers with point from stack to shared memory and this distance will be different in 32 bit (max. 32 bit distance) and 64 systems (max. 64 bit distance!). The pointer type is the same, so I'm afraid option 3 is not possible.
Option 2 suffers from the same problem. You can make pointers 32 bit but again 32 bit distance from stack to shared memory might be too short in 64 bit environments.
For options 1, the OS won't let you allocate/connect to memory bigger than 32 bit limits, so you will get an error when connecting/creating such shared memory.
Option 1: Say the requested size is 0x101000000 (4GB+16MB), for example. 32 bit OS would truncate that to 0x1000000 and all allocation/memory mapping calls would succeed (if you have 16MB to spare). Other internal datastructures being 64 bit, they would be set up to show the full 4GB+16MB. This wouldn't work. Option 3: The offset_ptr itself must be 64 bit, no question. But its size_type/difference_type do not have to be able to hold the difference between two arbitrary pointers, just between two pointers to the same memory block, which is max 32 bit: C++0x draft http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2010/n3225.pdf: 5.7.6 When two pointers to elements of the same array object are subtracted, the result is the difference of the subscripts of the two array elements. ... Unless both pointers point to elements of the same array object, or one past the last element of the array object, the behavior is undefined. 18.2 Types: 5 The type ptrdiff_t is an implementation-defined signed integer type that can hold the difference of two subscripts in an array object, as described in 5.7. 6 The type size_t is an implementation-defined unsigned integer type that is large enough to contain the size in bytes of any object. Likewise, in 20.2.5 Allocators: X::size_type a type that can represent the size of the largest object in the allocation model. X::difference_type a type that can represent the difference between any two pointers in the allocation model. The difference_type definition does not explicitly refer to 5.7.6 "Unless both pointers point to elements of the same array object, ... the behavior is undefined.", but I think that is intended. What do you think? Arno -- Dr. Arno Schödl | aschoedl@think-cell.com Technical Director think-cell Software GmbH | Chausseestr. 8/E | 10115 Berlin | Germany http://www.think-cell.com | phone +49 30 666473-10 | US phone +1 800 891 8091 Amtsgericht Berlin-Charlottenburg, HRB 85229 | European Union VAT Id DE813474306 Directors: Dr. Markus Hannebauer, Dr. Arno Schoedl
El 24/02/2011 15:10, Arno Schödl escribió:
Hello Ion, Option 1: Say the requested size is 0x101000000 (4GB+16MB), for example. 32 bit OS would truncate that to 0x1000000 and all allocation/memory mapping calls would succeed (if you have 16MB to spare). Other internal datastructures being 64 bit, they would be set up to show the full 4GB+16MB. This wouldn't work.
I can't see how a 32 bit OS could request 4016MB without detecting the overflow. The managed_shared_memory interface should take std::size_t as size. If the user truncates a 64 bit value to size_t the compiler should issue a warning, it's not interprocess' problem. When connecting to an existing shared memory, the OS should detect the overflow and issue an error if trying to connect a shared memory bigger than 32 bit. mapped_region does this for windows but the check is missing in UNIX (we need to check the return of fstat()-> st_size to be less than the maximum value of std::size_t). I'll add this.
Option 3: The offset_ptr itself must be 64 bit, no question. But its size_type/difference_type do not have to be able to hold the difference between two arbitrary pointers, just between two pointers to the same memory block, which is max 32 bit:
Ok, I missed that. I guess that if shared memory is less than 32 bits, then this would work. I think we should support both approaches: we should be able to use a 64 bit wide smart pointer that can define size_type and difference_type with different types (I guess we could impose size_type to be the unsigned version of difference_type). Offset_ptr should be the customization point for addressing options: memory algorithms and allocators get the addresing configuration from the pointer type, and containers should take the configuration from allocators. Thanks we have templates in the language ;-) Best, Ion
I can't see how a 32 bit OS could request 4016MB without detecting the overflow. The managed_shared_memory interface should take std::size_t as size. If the user truncates a 64 bit value to size_t the compiler should issue a warning, it's not interprocess' problem. When connecting to an existing shared memory, the OS should detect the overflow and issue an error if trying to connect a shared memory bigger than 32 bit. mapped_region does this for windows but the check is missing in UNIX (we need to check the return of fstat()-> st_size to be less than the maximum value of std::size_t). I'll add this.
Actually, it would try to allocate 4096MB+16MB, truncated to 16MB, which would succeed. I think any shared memory user either has to support 32 bit clients, and thus has to accommodate them, or not. Then compile-time checks and more compact structures due to smaller size_type/difference_type are better than runtime errors. We will BOOST_STATIC_ASSERT that offset_ptr::difference_type cannot be larger than ptrdiff_t. You can allow the runtime choice if you see the need. I currently don't.
Option 3: The offset_ptr itself must be 64 bit, no question. But its size_type/difference_type do not have to be able to hold the difference between two arbitrary pointers, just between two pointers to the same memory block, which is max 32 bit:
Ok, I missed that. I guess that if shared memory is less than 32 bits, then this would work.
we should be able to use a 64 bit wide smart pointer that can define size_type and difference_type with different types (I guess we could impose size_type to be the unsigned version of difference_type). Offset_ptr should be the customization point for addressing options: memory algorithms and allocators get the addresing configuration from the pointer type, and containers should take the configuration from allocators. Thanks we have templates in the language ;-)
That's exactly what we did. The unit tests pass. We'll do some more checks, also looking at the compiler warnings for truncation so we can justify every lossy cast we added (very few), and then you'll get the patch. Thanks! Arno -- Dr. Arno Schödl | aschoedl@think-cell.com Technical Director think-cell Software GmbH | Chausseestr. 8/E | 10115 Berlin | Germany http://www.think-cell.com | phone +49 30 666473-10 | US phone +1 800 891 8091 Amtsgericht Berlin-Charlottenburg, HRB 85229 | European Union VAT Id DE813474306 Directors: Dr. Markus Hannebauer, Dr. Arno Schoedl
El 24/02/2011 18:46, Arno Schödl escribió:
That's exactly what we did. The unit tests pass. We'll do some more checks, also looking at the compiler warnings for truncation so we can justify every lossy cast we added (very few), and then you'll get the patch.
Ok, sounds pretty good. Thanks for your help! Ion
participants (2)
-
Arno Schödl -
Ion Gaztañaga