On Wed, 2005-05-04 at 08:52 -0700, Nathan Myers wrote:
On Wed, May 04, 2005 at 02:46:05PM +0100, Iain K. Hanson wrote:
On Tue, 2005-05-03 at 20:55 -0700, Nathan Myers wrote:
On Wed, May 04, 2005 at 09:07:24AM +1200, Scott Woods wrote:
From: "Nathan Myers" <ncm@cantrip.org>
(Another goal is a zero-copy streambuf whose buffer is an mmap page that can be read into or written from without actually copying any bytes from kernel to user space, or back.
That would be impressive! Unfortunately I don't think it is theoretically possible ;-). Under normal conditions the interface device driver copies from the hardware device buffer to kernel space. The data passes up the stack until it is copied from kernel space to user space for a total of 2 copies. you would also get two copies on the other side ( mmap ). It is certainly possible to eliminate one copy on each side.
Copying from the device to kernel memory is unavoidable, in general, but typically only costs bus bandwidth, not CPU cycles. The copy from kernel to user space can be avoided if the user's buffer is an anonymously-mmap()ped page that hasn't been written in, because one page is as good as another.
I don't know sufficient about the implementation of mmap and its relationship with swap system and buffer cache and the IP stack buffers to know whether this would work as you intend.
Similarly for output -- although of course it's OK to have written it! Then, though, if it _hasn't_ been touched, it might be a kernel buffer that the user code only inspected (or didn't) and just wants passed along. (That's how NetBSD UVM works, IIUC.)
You will still at a minimum have another kernel to device copy as previously stated. Another problem is that mmap files *I think* need to be seek'able.
I believe Doug Schmidt has done something on this but just on sockets. IIRC it involved writing his own device drive and use on Data Link Layer access to sockets to copy directly from the hardware buffer to user space. The down side of this is you now need your IP stack also in user space.
An IP stack in user space would kill latency and probably throughput, obviating gains from avoiding copies. But a cool project anyway. Maybe Hurd can use it. :-)
I think that depends on the process priority and what else is running on the system. But certainly a very interesting concept ;-).
Boost.iostream seems to have something like I described, allowing the parsing of regular files with zero-copy. Probably that ought to be slurped into GNU libstdc++ filebuf proper. However, having it in Boost (too) makes it portable, e.g. to older g++ releases.
I have not yet looked at boost iostreams but it is on my todo list :-(. /ikh _______________________________________________________________________ This email has been scanned for all known viruses by the MessageLabs Email Security System. _______________________________________________________________________