Hello, ----- Original Message ----

From: Stuart Dootson

...

I am looking for a memory mapped file based allocator that does not map the whole file into memory. The size may be huge and I would not want to map all the

One solution (which I've used with files of gigabyte size) that you could consider is writing a class to encapsulate memory mapped files that exposes the file contents only through a custom iterator class. These could work together to map and unmap portions of the file as the iterator position crosses 'page' boundaries. This would give you something close to on-demand paging.

Thank you for the suggestion. I have implemented it on windows using SEH (structured exception handling). Is there something similar in the UNIX world. Adding a signal handler will be for the whole process, I need to check if there is a way to add a signal handler for a specific thread that is doing the read/write and perform the unmap and map sequence to keep moving the read/write window. This would be a great feature to have it tucked somewhere in BOOST. Reading/writing large files is quite common. If I succeed in putting in a solution that works on M$ and NIX world, I will post it. -dhruva Connect with friends all over the world. Get Yahoo! India Messenger at http://in.messenger.yahoo.com/?wm=n/

On Wed, Mar 11, 2009 at 1:20 AM, Stuart Dootson wrote:

On Wed, Mar 11, 2009 at 3:41 AM, dhruva wrote:

...

----- Original Message ----

...

From: Stuart Dootson

...

 I am looking for a memory mapped file based allocator that does not map the whole file into memory. The size may be huge and I would not want to map all the

...

One solution (which I've used with files of gigabyte size) that you could consider is writing a class to encapsulate memory mapped files that exposes the file contents only through a custom iterator class. These could work together to map and unmap portions of the file as the iterator position crosses 'page' boundaries. This would give you something close to on-demand paging.

Thank you for the suggestion. I have implemented it on windows using SEH (structured exception handling). Is there something similar in the UNIX world. Adding a signal handler will be for the whole process, I need to check if there is a way to add a signal handler for a specific thread that is doing the read/write and perform the unmap and map sequence to keep moving the read/write window. This would be a great feature to have it tucked somewhere in BOOST. Reading/writing large files is quite common. If I succeed in putting in a solution that works on M$ and NIX world, I will post it.

-dhruva

You don't need to use exceptions at all - in the iterator's 'dereference' method (see the iterator_facade documentation in Boost.Iterator for what that means), check if the item that the iterator references is in the currently mapped region. If not, map a region that does contain that item.

I suppose the motivation for using structured exceptions is that you don't have to do this check, which is presumably faster. But structured exception handling itself introduces overhead (compared to "regular" exception handling) so usually it is not a good idea to require it. Emil Dotchevski Reverge Studios, Inc. http://www.revergestudios.com/reblog/index.php?n=ReCode