Jonathan Turkanis wrote:
Roman Dementiev wrote:
Jonathan Turkanis wrote:
Roman Dementiev wrote:
The Boost filesystem library is more about manipulating files and directories. But what I need is the file access itself: create/open/read/write/close file. I would have been using std::fstream as a portable file access method, but it lacks support of files larger than 2 GB. It is really big disadvantage in my case.
The boost iostreams library, which is in CVS now and will be part of the next boost release, contains a class file_descriptor for accessing files using OS or runtime library file descriptors.
Under Linux if preprocessor macros _LARGEFILE_SOURCE, _LARGEFILE64_SOURCE, _FILE_OFFSET_BITS=64 are defined then off_t is 64 bit, otherwise it is 32 bit.
Access via iostreams will still have to use std::streamoff, since that is out of my control; but you should be able to use an instance of file_descriptor directly.
The new code should be available soon.
I think boost::iostreams::file_descriptor is what I need. Of course with 64 bit seek.
Okay, I'll make sure to add this.
Another reason why I do not like std::fstream is its bad performance. It introduces a superfluous copying into it's internal buffer when doing I/O on user buffer.
You can also use a FILE* and specify _IONBF with setvbuf.
Another issue is that the performance of my library will benefit from unbuffered file system access. Many operating system support it (O_DIRECT open option in Unix systems,
FILE_FLAG_NO_BUFFERING|FILE_FLAG_WRITE_THROUGH option for "CreateFile"
Windows API call).
Could you point me to documentation for O_DIRECT? man 2 open http://www.die.net/doc/linux/man/man2/open.2.html http://www.mcsr.olemiss.edu/cgi-bin/man-cgi?open+2
It looks like FILE_FLAG_NO_BUFFERING requires some care to use properly, so I think it would have to go into a specialized component instead of being added to file_descriptor. Do you mean the offset requirements described in http://msdn.microsoft.com/library/default.asp?url=/library/en-us/fileio/base... ?
In particular Stxxl file access patterns meet these requirements. Roman