Boost policy on relying on std library implementation defined behaviour
Hi All, Is there a reason to avoid providing a uniform interface to implementation-defined standard library features? One of the most often heard complaints about the standard iostreams library is that there is no portable way to seek within a file stream using large (64-bit) offsets. Simplifying a bit, the standard library provides functions for seeking within a stream using - an offset of type streamoff, which may be 64-bits but is sometimes smaller, or - a position indicator of type streampos (usually fpos<mbstate_t>), a non-integral implementation defined type, with implicit conversions to and from streamoff. (streampos also stores a code conversion state, but I'm ignoring this) There are platforms where streamoff is inadequate to express an arbitrary seek offset, and where streampos is adequate, but only if implementation defined functionality is used. (There might be cases where neither type is adequate, but I'm ignoring this) I would like to implement functions which provide conversions between a large integral type and streampos. This will allow most uses of streampos to be hidden, with library users specifying integral types most of the time. The alternative would be: 1. provide additional concepts which require user defined filters and devices to implement functions involving streampos, AND 2. force users to deal with streampos directly, relying whatever guarantees are provided by their standard libraries. Point 1. requires complicating the iostreams library interface. Point 2. forces users to implement functions which could instead have been implemented just once by the iostreams library. What do people think? Jonathan
"Jonathan Turkanis" <technews@kangaroologic.com> wrote in message news:d1v087$6r0$1@sea.gmane.org...
Hi All,
Is there a reason to avoid providing a uniform interface to implementation-defined standard library features?
One of the most often heard complaints about the standard iostreams library is that there is no portable way to seek within a file stream using large (64-bit) offsets.
I really want to see this fixed in the standard library. Any interest in getting a proposal together? It seems to me the first step is to fix the <cstdio> stuff inherited from the C library. An easy way to do that is to propose the addition of fseeko and ftello functions. They are already provided on Solaris, FreeBSD, HP-UX, NetBSD, OpenBSD, and HP/Compaq/DEC Alpha OSF/1. SGI IRIX offers the same functionality, but named fseek64() and ftell64(). Apparently Linux is the only weak spot in the POSIX world. The functionality is trivial to provide under modern flavors of Windows. (Thanks to Nelson Beebe for survey of POSIX practice.) I haven't given much thought to changes that would be required for iostreams yet. --Beman
Beman Dawes wrote:
"Jonathan Turkanis" <technews@kangaroologic.com> wrote in message news:d1v087$6r0$1@sea.gmane.org...
Hi All,
Is there a reason to avoid providing a uniform interface to implementation-defined standard library features?
One of the most often heard complaints about the standard iostreams library is that there is no portable way to seek within a file stream using large (64-bit) offsets.
I really want to see this fixed in the standard library. Any interest in getting a proposal together?
It seems to me the first step is to fix the <cstdio> stuff inherited from the C library. An easy way to do that is to propose the addition of fseeko and ftello functions. They are already provided on Solaris, FreeBSD, HP-UX, NetBSD, OpenBSD, and HP/Compaq/DEC Alpha OSF/1. SGI IRIX offers the same functionality, but named fseek64() and ftell64(). Apparently Linux is the only weak spot in the POSIX world.
Hmm.. How is Linux the weak spot? It does provide lseek64 and fseeko under certain conditions, and ftell64 is implementable as lseek64(fd,0,SEEK_CUR). -- -- Grafik - Don't Assume Anything -- Redshift Software, Inc. - http://redshift-software.com -- rrivera/acm.org - grafik/redshift-software.com - 102708583/icq
"Rene Rivera" <grafik.list@redshift-software.com> wrote in message news:42449929.1090101@redshift-software.com...
Beman Dawes wrote:
Apparently Linux is the only weak spot in the POSIX world.
Hmm.. How is Linux the weak spot? It does provide lseek64 and fseeko under certain conditions, and ftell64 is implementable as lseek64(fd,0,SEEK_CUR).
That's good to hear. I was quoting the Bebee survey - perhaps it was made some time ago. I just checked Apple, and they also supply the functionality. Anyhow, it is hard to see how there could be much opposition to standardizing something where there is so much existing practice. That leaves <iostreams>. There are a few functions specified in terms of fseek. It should be trivial to change those descriptions to specify in terms of fseeko, and adjust argument types accordingly. But there are some other cases I'm foggy about. --Beman
Beman Dawes wrote:
That leaves <iostreams>. There are a few functions specified in terms of fseek. It should be trivial to change those descriptions to specify in terms of fseeko, and adjust argument types accordingly. But there are some other cases I'm foggy about.
You remind me that I forgot to propose changes to 27.8. The obvious change is to basic_filebuf::seekoff; instead of calling fseek it should call fseeko. Another possible change might be to the specification of the effect of the openmode ios_base::ate (27.8.1.3/3), though I'm not sure any change is necessary. Jonathan
Beman Dawes wrote:
"Jonathan Turkanis" <technews@kangaroologic.com> wrote in message news:d1v087$6r0$1@sea.gmane.org...
Hi All,
Is there a reason to avoid providing a uniform interface to implementation-defined standard library features?
One of the most often heard complaints about the standard iostreams library is that there is no portable way to seek within a file stream using large (64-bit) offsets.
I really want to see this fixed in the standard library. Any interest in getting a proposal together?
Yes, definitely.
It seems to me the first step is to fix the <cstdio> stuff inherited from the C library. An easy way to do that is to propose the addition of fseeko and ftello functions. They are already provided on Solaris, FreeBSD, HP-UX, NetBSD, OpenBSD, and HP/Compaq/DEC Alpha OSF/1. SGI IRIX offers the same functionality, but named fseek64() and ftell64(). Apparently Linux is the only weak spot in the POSIX world. The functionality is trivial to provide under modern flavors of Windows. (Thanks to Nelson Beebe for survey of POSIX practice.)
Good survey.
I haven't given much thought to changes that would be required for iostreams yet.
Two possibilities I can think of are as follows (I'm writing this off the top of my head -- I'll need to give it more thought): 1. require streamoff (and the various member types, such as char_traits<>::off_type which have the same requirements as streamoff) to satisfying whatever requirements you would impose on the second parameter of fseeko. 2. Change the legend to Table 88 (27.4.3.2/1) so that - instead of referring to type streamoff, "O" refers to an integral type satisfying whatever requirements you would impose on the second parameter of fseeko. - "o" refers to a value of type "O" Option 1 is more straightforward, but could require some standard libraries (notably Dinkumware) to change the size of their streamoff type even though their streampos already provides adequate large-file support. Standard libraries which already provide a large streamoff would need no change. Option 2 would allow standard libraries which provide a small streamoff but a streampos capable of storing large values to conform by providing a streampos constructor callable with a single parameter of type "O" and by replacing the conversion to streamoff with a conversion to "O". How do these strike you? Jonathan
"Jonathan Turkanis" <technews@kangaroologic.com> skrev i meddelandet news:d22hfp$7e9$1@sea.gmane.org...
Two possibilities I can think of are as follows (I'm writing this off the top of my head -- I'll need to give it more thought):
1. require streamoff (and the various member types, such as char_traits<>::off_type which have the same requirements as streamoff) to satisfying whatever requirements you would impose on the second parameter of fseeko.
2. Change the legend to Table 88 (27.4.3.2/1) so that - instead of referring to type streamoff, "O" refers to an integral type satisfying whatever requirements you would impose on the second parameter of fseeko. - "o" refers to a value of type "O"
Option 1 is more straightforward, but could require some standard libraries (notably Dinkumware) to change the size of their streamoff type even though their streampos already provides adequate large-file support. Standard libraries which already provide a large streamoff would need no change.
Option 2 would allow standard libraries which provide a small streamoff but a streampos capable of storing large values to conform by providing a streampos constructor callable with a single parameter of type "O" and by replacing the conversion to streamoff with a conversion to "O".
How do these strike you?
Jonathan
Doesn't this run into the same problems as the original standard does - what to do when there isn't an integral type large enough? The current library definition uses type long, because that is as wide as it gets! Bo Persson
"Bo Persson" <bop@gmb.dk> wrote in message news:d23cl3$sp4$1@sea.gmane.org...
...
How do these strike you?
Jonathan
Doesn't this run into the same problems as the original standard does - what to do when there isn't an integral type large enough? The current library definition uses type long, because that is as wide as it gets!
Yes, but there are two ways around the problem. (1) Use long long. Almost all late-model C++ compilers already support long long, and the C++ committee is in process of adding it to the language. (2) Make it an implementation-defined type which must be able to hold the largest value the operating system supports. That means int or long is fine if sufficient for the platform. It also means that a type like __int64 could be used, or even long long itself. If the C++ committee's library working group (LWG) prefers to the additions go in C++0x, then (1) is fine. If the additions go in the second library technical report (TR2), then (2) might be better. --Beman
On Sat, 26 Mar 2005 08:54:42 -0500, Beman Dawes wrote
Yes, but there are two ways around the problem. (1) Use long long. Almost all late-model C++ compilers already support long long, and the C++ committee is in process of adding it to the language. (2) Make it an implementation-defined type which must be able to hold the largest value the operating system supports. That means int or long is fine if sufficient for the platform. It also means that a type like __int64 could be used, or even long long itself.
Before long long is standard you can use boost::int64_t takes care of the portability issues of long long versus __int64 for all the platforms we are currently supporting. Jeff
"Jonathan Turkanis" <technews@kangaroologic.com> wrote in message news:d22hfp$7e9$1@sea.gmane.org...
Two possibilities I can think of are as follows (I'm writing this off the top of my head -- I'll need to give it more thought):
1. require streamoff (and the various member types, such as char_traits<>::off_type which have the same requirements as streamoff) to satisfying whatever requirements you would impose on the second parameter of fseeko.
2. Change the legend to Table 88 (27.4.3.2/1) so that - instead of referring to type streamoff, "O" refers to an integral type satisfying whatever requirements you would impose on the second parameter of fseeko. - "o" refers to a value of type "O"
Option 1 is more straightforward, but could require some standard libraries (notably Dinkumware) to change the size of their streamoff type even though their streampos already provides adequate large-file support. Standard libraries which already provide a large streamoff would need no change.
Option 2 would allow standard libraries which provide a small streamoff but a streampos capable of storing large values to conform by providing a streampos constructor callable with a single parameter of type "O" and by replacing the conversion to streamoff with a conversion to "O".
How do these strike you?
I started to answer your question as asked, but then realized it isn't my opinion that counts, but rather the LWG's collective opinion, which will be heavily influenced by the implementers. So maybe what we ought to do over the next two weeks is to build up a list of questions, then ask the LWG in Lillehammer for their preferences. Action plan for the next couple of days is to read chapter 27, and try to identify as many issues as possible. --Beman
Beman Dawes wrote:
"Jonathan Turkanis" <technews@kangaroologic.com> wrote in message news:d22hfp$7e9$1@sea.gmane.org...
Two possibilities I can think of are as follows (I'm writing this off the top of my head -- I'll need to give it more thought):
How do these strike you?
I started to answer your question as asked, but then realized it isn't my opinion that counts, but rather the LWG's collective opinion, which will be heavily influenced by the implementers.
Sounds reasonable (but I can't help wondering what your own opinion was ;-) )
So maybe what we ought to do over the next two weeks is to build up a list of questions, then ask the LWG in Lillehammer for their preferences.
Unfortunately, my plans have changed and I won't be in Lillehammer, but I can still help you draw up the list.
Action plan for the next couple of days is to read chapter 27, and try to identify as many issues as possible.
Sounds good.
--Beman
Jonathan
participants (6)
-
Beman Dawes -
Bo Persson -
David Abrahams -
Jeff Garland -
Jonathan Turkanis -
Rene Rivera