On Wednesday, May 3, 2006 at 07:53:15 (-0700) Robert Ramey writes:
... Since you don't have 60,000 functions in your program, that would almost surely indicate a stack overflow due to un-terminated recursion. Just look at the stack back from the end the recursion should be obvious.
After plowing through endless reams of data and stack frames, looking for an obnoxious pattern, I finally tried what should have been obvious: check how much stack size my processes are allowed and try bumping that. Surely enough, I was granted 10 megabytes of stack size. I removed the ceiling on this (ulimit -s unlimited) and the thing worked, repeatedly, no problems. So, it appears that these larger data sets are just pressing the stack quite hard, so boost is apparently not the culprit. I seem to remember, though, seeing tons frames of boost library calls for a single serialization output --- seemingly lots of "dispatch" type template redirections. Is there any way to optimize some of this out? I suppose not, but just wanted to ask. In the course of this little adventure, I did come across something else that I'll try to put into a more thoughtful email later: exceptions in the serialization library are sometimes maddeningly unhelpful (C++ exceptions in general drive me nuts sometimes). One time, I successfully wrote the entire structure out to disk, and after the routine had returned, boost threw a "stream error" --- I finally tried setting ios::binary on the stream and this seemed to fix it. But the "stream error" had no context, obviously the code just looks for the fail bit being set (also the bad bit, I think), and just throws an exception. But, there is no context to this, no stack trace (C++ exception), etc. I did notice that in gdb you can put a debugger "catch" on a C++ throw --- next time I get a stream error, I will try this trick. Anyway, thank you again for all of your help. I am tired, but happy that the boost library doesn't have some sort of weird bug and that this was all just a case of me racing down the non-obvious path, instead of choosing the one (seemingly, always) less-traveled by. Bill