[btree] Large file support
The btree library uses low level I/O. On POSIX-like systems that means read/write/lseek. lseek() won't support large files on systems where long is 32 bits. POSIX-like operating systems get around this problem by supply functions with names like llseek or lseek64 that traffic in long long instead of long. So there is a line of code: new_offset = ::lseek(handle(), offset, whence); // offset is a variable of type int64_t That needs to change to something like: #if defined(SOMEOS) new_offset = ::|lseek(handle(), offset, whence); #elif defined(ANOTHEROS) new_offset = ::whatever(handle(), offset, whence); #else new_offset = ::lseek(handle(), offset, whence); #endif Up at the front of the source file there may also have to be special headers included or symbols defined, depending on the operating system. I'd appreciate some help from folks familiar with particular non-Windows operating systems. What is the preferred name for the 64-bit seek function and what is the preferred #if defined(symbol)? Do special headers have to be included or special symbols defined? Thanks, --Beman
Beman Dawes wrote:
POSIX-like operating systems get around this problem by supply functions with names like llseek or lseek64 that traffic in long long instead of long.
So there is a line of code:
new_offset = ::lseek(handle(), offset, whence); // offset is a variable of type int64_t
That needs to change to something like:
#if defined(SOMEOS) new_offset = ::|lseek(handle(), offset, whence); #elif defined(ANOTHEROS) new_offset = ::whatever(handle(), offset, whence); #else new_offset = ::lseek(handle(), offset, whence); #endif
When code isn't separated by platform, and semantics are the same, I prefer to define a macro like this: #ifdef SOMEOS # define LSEEK ::lseek #else # define LSEEK ::lseek64 #endif Then the later invocation is just this: new_offset = LSEEK(handle(), offset, whence); If the macro would be exposed in a header, then a more unique name would be appropriate, of course.
Up at the front of the source file there may also have to be special headers included or symbols defined, depending on the operating system.
I'd appreciate some help from folks familiar with particular non-Windows operating systems. What is the preferred name for the 64-bit seek function and what is the preferred #if defined(symbol)? Do special headers have to be included or special symbols defined?
I'm using lseek64() in my 64b code. According to the manpages, the required includes don't differ between lseek() and lseek64(). The latter is only available when _LARGEFILE64_SOURCE is defined. The manpage goes on to say that lseek64() is an alias for llseek() and has been available since glibc 2.1. However, when I look up llseek(2), it redirects to _llseek(2) and a search of headers in /usr/include doesn't turn up a .h with llseek(). That suggests lseek64() is a safer name. FYI: lseek() uses a 32b off_t on 32b platforms unless _FILE_OFFSET_BITS is #defined as 64. In that case, lseek() is the same as lseek64(). _____ Rob Stewart robert.stewart@sig.com Software Engineer, Core Software using std::disclaimer; Susquehanna International Group, LLP http://www.sig.com IMPORTANT: The information contained in this email and/or its attachments is confidential. If you are not the intended recipient, please notify the sender immediately by reply and immediately delete this message and all its attachments. Any review, use, reproduction, disclosure or dissemination of this message or any attachment by an unintended recipient is strictly prohibited. Neither this message nor any attachment is intended as or should be construed as an offer, solicitation or recommendation to buy or sell any security or other financial instrument. Neither the sender, his or her employer nor any of their respective affiliates makes any warranties as to the completeness or accuracy of any of the information contained herein or that this message or any of its attachments is free of viruses.
On Fri, Oct 1, 2010 at 1:59 PM, Beman Dawes <bdawes@acm.org> wrote:
The btree library uses low level I/O. On POSIX-like systems that means read/write/lseek. lseek() won't support large files on systems where long is 32 bits.
POSIX-like operating systems get around this problem by supply functions with names like llseek or lseek64 that traffic in long long instead of long.
[...]
Up at the front of the source file there may also have to be special headers included or symbols defined, depending on the operating system.
I'd appreciate some help from folks familiar with particular non-Windows operating systems. What is the preferred name for the 64-bit seek function and what is the preferred #if defined(symbol)? Do special headers have to be included or special symbols defined?
IIRC that today posix/SUSv4 simply declares off_t lseek(int fildes, off_t offset, int whence); Unistd.h has the following macros: _XBS5_ILP32_OFF32 Implementation provides a C-language compilation environment with 32-bit int, long, pointer and off_t types. _XBS5_ILP32_OFFBIG Implementation provides a C-language compilation environment with 32-bit int, long and pointer types and an off_t type using at least 64 bits. _XBS5_LP64_OFF64 Implementation provides a C-language compilation environment with 32-bit int and 64-bit long, pointer and off_t types. _XBS5_LPBIG_OFFBIG Implementation provides a C-language compilation environment with an int type using at least 32 bits and long, pointer and off_t types using at least 64 bits. reference (and more) here: http://opengroup.org/onlinepubs/007908799/xsh/unistd.h.html llseek and lseek are non portable remainants of the move to large files. Most unix systems should have moved to the standard interface. Usually macros are available to map the lseek interface to the non portable internal interface. I think linux uses _FILE_OFFSET_BITS. HTH, -- gpd
non-Windows operating systems. What is the preferred name for the 64-bit seek function and what is the preferred #if defined(symbol)?
I'd recommend having a look at how Boost.Iostreams manages this. See libs/iostreams/src/file_descriptor.cpp and boost/iostreams/detail/config.rtl.hpp, e.g. BOOST_IOSTREAMS_FD_SEEK macro. Best regards, Gareth ************************************************************************ The information contained in this message or any of its attachments may be confidential and is intended for the exclusive use of the addressee(s). Any disclosure, reproduction, distribution or other dissemination or use of this communication is strictly prohibited without the express permission of the sender. The views expressed in this email are those of the individual and not necessarily those of Sony or Sony affiliated companies. Sony email is for business use only. This email and any response may be monitored by Sony to be in compliance with Sony's global policies and standards
participants (4)
-
Beman Dawes -
Giovanni Piero Deretta -
Stewart, Robert -
Sylvester-Bradley, Gareth