21 Mar
2010
21 Mar
'10
1:35 a.m.
Phil Endecott wrote:
However, recently I needed something that used less memory. I wanted to process a very large file without ever having all of it in memory (imagine e.g. loading a database). So I wrote something where the element and attribute iterators (etc.) are just pointers into the (memory-mapped) XML source. When an iterator is incremented it steps through the source looking (textually) for the start of the next element or attribute (etc.). The result is something that uses almost no memory and is fast for the sorts of access pattern that I needed.
This is exactly the kind of approach I find interesting. Could you share this code?