At 9:25 PM +0100 3/10/06, Ion Gaztañaga wrote:
You can use MSDN (www.msdn.com)
Thanks for the pointer.
A POSIX shared memory object exists from the time it is created until it is unlinked and all references are gone (or the system is rebooted). (It becomes inaccessible to further shm_open calls if unlinked, but remains open to processes that had already opened it.)
Right. The difference is that in windows, if there's still a process connected to the shared memory, another process can open the same segment. If a UNIX process calls shm_unlink, them the processes that opened the shared memory can still work on it, but if another process can't use that memory, because the name is removed.
The other difference is that on Windows, if there are no processes connected to the shared memory, then it goes away. For both POSIX and SysV shared memory, once created, a shared memory exists until unlinked (or system reset). So a process can create and put some stuff in shared memory, and then exit normally, and sometime later another process can come along and still find that same shared memory data. That doesn't work on Windows (or with the current reference counting implementation of the library).
So if a process calls shm_unlink another process can create new shared memory segment with the same name, but never connect to that old shared memory. This is different from a file unlink behavior, I think.
No, this is actually the same a a file unlink behavior. Intentionally so, so that an implementation can basically use files (possibly in a special file system) to implement shared memory. If several processes all have opened a normal file, and that file is then deleted from the file system, the processes still retain their handles on the file and can perform file-io operations on it. Internally the file handles are attached to the inode, which doesn't get reclaimed until there are no references (including from the filesystem).
What the present library implementation is trying to do with this reference count mechanism is to emulate the Windows behavior on POSIX systems.
Right. I find windows behaviour easier to implement with posix (when there is no crash) than emulating posix with windows.
Yes, I can't think of a better way to implement the POSIX behavior on Windows than your suggestions of either memory mapped files or a server program to act as a proxy for the POSIX create/unlink operations. The proxy server is particularly not pretty.
I'm sure there is a solution.
I wish :( There almost could be, by using a server program on POSIX systems. Rough sketch: open connection to server, tell server about opens and closes. When connection is closed, server fixes reference counts for any shared memory opens not yet closed. select() can be used to get notification of connection closes even for crashes. I haven't thought this through carefully, so there may be race conditions in there.
One option is to use a memory mapped file to emulate posix-like behavior in windows. But it surely will be slower (the OS will dump data to file) and the unlink is not trivial.
That does seem like it would be bad if true. Are there any file manipulation options that might be used to control that? I don't know, but I'll ask around and do some reading and see what I can find. [Later] As long as you don't flush the file it would seem like a memory mapped file should be no different from a shared memory in performance, the only difference is where data gets written if it needs to be swapped out. The big downside to using an actual file would be that you need a place in the file system to put it. Then you get into the whole name of the shared memory portability problem again.
Windows natively does not support posix nor System V. I think that with Unix Services for Windows (previously Interix)you have system V interface. I think Unix Services for Unix is a free download now, but I think you need to use that environment to build a Unix-like application (using gcc compiler). I don't know if those system V functions are available with a normal windows environment.
Yeah, I did some more research, and that matches what I found. That path looks like a good way to ensure that Windows programmers don't use the library. Oh well... But wait. We don't need to use SysV shared memory to get reference counting behavior on Windows. So if reference counting behavior is indeed what the library will provide, Windows native API's provide it, and (most or perhaps all) Unix ports could use the SysV API, though as noted in my earlier message, that may have some issues too, regarding kernel limits. For example, I figured out where to find the information for OSX and (by default) you only get 4M maximum size, 32 total, 8 per process. [Note that my own preference is toward not reference counting, but the potential impact on Windows is certainly an issue.]