
I have, on occasion, had a need to read large files. Sometimes, boost::iostreams::mapped_file works just fine, as long as I have enough memory space available to map the file in its entirety. Sometimes I don't. I have been thinking of making io::mapped_file support the notion of memory-mapped files with sliding windows. I have tried a little bit of code, but without much success because I hadn't thought through the design very well (and part of that I blame on my current position on the learning curve of developing boost libraries). In particular, I tried adding a couple of parameters to the mapped_file_params struct, and then coding support for it in the mapped_file_source class. This doesn't get very far in getting what I want for several reasons mostly because it doesn't address how the upper layers of iostreams will use the device. The data(), begin(), and end() member functions return simple char* (the iterator type is just typedef char *). I'd like to take another approach, which I think would be better than my initial attempt. I'd like to do something approximating the following: template<class W = default_window_manager> class mapped_file_source { public: typedef W::iterator iterator_type; // ... iterator_type begin() const; iterator_type end() const; iterator_type data() const; // ... private: W _manager; }; Where 'class W' would have to model something like the following: class window_manager { public: struct iterator { operator++() { if(current_ptr+1 >= current_map_end) // remap file to the next block, re-adjust pointers } operator--() { if(current_ptr-1 <= current_map_begin) // remap file to the previous block, re-adjust pointers } // operators for +,- might also need to do something non-trivial }; private: // member functions to perform re-mapping current_ptr; current_map_begin; current_map_end; }; Of course, the above does not deal with random access at all. 'default_window_manager' could just provide a windowed memory map with a single window that is exactly the size of the file to make mapped_file behave as it currently does. It seems to me that this could be generally useful, but perhaps reality is that I need to be saved from myself. Comments? I apologize in advance if this is has been hashed over before. My searches of the archives turned up nothing, but the internet is too big a place to prove a negative. -- Benjamin A. Collins <ben.collins@acm.org> http://bloggoergosum.us