[interprocess] semantics of mmap()d files on network filesystems
Dear All, Does anyone have any ideas about the behaviour of memory-mapped files on network filesystems, i.e. NFS and/or Windows shares? Is there any chance that Boost.Interprocess could use mmaped files on such filesystems, with instances of the program running on different machines sharing data? Regards, Phil.
Phil Endecott wrote:
Does anyone have any ideas about the behaviour of memory-mapped files on network filesystems, i.e. NFS and/or Windows shares? Is there any chance that Boost.Interprocess could use mmaped files on such filesystems, with instances of the program running on different machines sharing data?
MSDN has this to say for CreateFileMapping:
Although *CreateFileMapping* works with remote files, it does not keep them coherent. For example, if two computers both map a file as writable, and both change the same page, each computer only sees its own writes to the page. When the data gets updated on the disk, it is not merged.
Although I couldn't find concrete information in the man pages, some Google results on "mmap consistent" suggest that mmap behaves the same. Sebastian Redl
Sebastian Redl wrote:
Phil Endecott wrote:
Does anyone have any ideas about the behaviour of memory-mapped files on network filesystems, i.e. NFS and/or Windows shares? Is there any chance that Boost.Interprocess could use mmaped files on such filesystems, with instances of the program running on different machines sharing data?
MSDN has this to say for CreateFileMapping:
Although *CreateFileMapping* works with remote files, it does not keep them coherent. For example, if two computers both map a file as writable, and both change the same page, each computer only sees its own writes to the page. When the data gets updated on the disk, it is not merged.
Any ideas when the write-back occurs? When the file is closed / unmapped? Can you call flush to force write back? (Does interprocess support any of this?)
Although I couldn't find concrete information in the man pages, some Google results on "mmap consistent" suggest that mmap behaves the same.
I have had a look at the NFS spec and it says a lot about what implementations "may" do, and it's similar. A simpler case than the example that you quote above is one writer and multiple readers. The main requirement is that when the writer's changes become visible to a reader they do so atomically, or at least in-order. I.e. if I have a large interprocess::list which spans multiple blocks, and the writer inserts something, then the readers must see either no changed blocks or all changed blocks. Or, if I have some sort of lock-free structure like a journal, and the writer appends data and then updates an end-of-journal pointer to make it visible, then the page containing the pointer must not become visible to the readers until after the pages containing the data. Anyway, I think the answer is that this is either hard or impossible; but I thought I'd ask just in case someone has already investigated. Regards, Phil.
Phil Endecott wrote:
Any ideas when the write-back occurs? When the file is closed / unmapped? Can you call flush to force write back? (Does interprocess support any of this?)
Interprocess uses OS dependent functions. For example, FlushViewOfFile for Windows states (http://msdn2.microsoft.com/en-us/library/aa366563.aspx): "When flushing a memory-mapped file over a network, FlushViewOfFile guarantees that the data has been written from the local computer, but not that the data resides on the remote computer. The server can cache the data on the remote side. Therefore, FlushViewOfFile can return before the data has been physically written to disk. However, you can cause FlushViewOfFile to return only when the physical write is complete by specifying the FILE_FLAG_WRITE_THROUGH flag when you open the file with the CreateFile function." The flag FILE_FLAG_WRITE_THROUGH is not used for performance reasons, so synchronization is not guaranteed.
Anyway, I think the answer is that this is either hard or impossible; but I thought I'd ask just in case someone has already investigated.
I'm afraid you are right :-( Regards, Ion
participants (3)
-
Ion Gaztañaga -
Phil Endecott -
Sebastian Redl