[boost] asio queue OOM stress test

16 Dec 2005

      I wrote a little program to stress the queue
under low memory situations, and to
demonstrate that there is no upper bound
on memory usage when using async function
calls.

The first time I ran it under Linux the kernel
killed mysqld and pretty much hung the system.  The
second time the kernel killed the test program.
In neither run could I recover from the OOM error,
which is one reason I wrote this.

I want to demonstrate that under Linux you often
can't catch exceptions from new, because your
program is dead before the exception is thrown.
Some might consider that a limitation of over
committing memory managers like Linux and FreeBSD's,
but that's how it works, and I think we are stuck
with it.

What is really interesting is on Windows, as
I expected, I did get the exception and started
running the handlers, but the program deadlocked
each time I ran it after running about 600k
function calls.  I'm starting to wonder if this
a bug unrelated to the OOM situation.

#include <iostream>

#include <boost/bind.hpp>
#include <boost/asio.hpp>

void doit(unsigned long* count)
{
  ++(*count);
  if(!(*count % 100000)){
    std::cout<<*count<<std::endl;
  }
}

int main(int argc, char* argv[])
{
  boost::asio::demuxer d;

  unsigned long call_count = 0;
  unsigned long post_count = 0;
  //
  // post messages until we run out memory and then
  // run them.
  try{
    std::cout<<"posting events..."<<std::endl;
    for(post_count = 1;;++post_count){
      //
      // Internally allocates queueing structures, and
      // a copy of the functor returned by bind.
      d.post(boost::bind(doit, &call_count));
      if(!(post_count % 100000)){
        //
        // print out something to show that
        // we are still alive.
        std::cout<<post_count<<std::endl;
      }
    }
  }
  catch(...){
    //
    // Let's not do anything here in case we throw again.
  }
  //
  // There is a reasonable chance we might throw here,
  // because we are pretty much out of memory.  Might
  // want to comment this line out.
  std::cout<<"caught exception. post_count: "
           <<post_count<<std::endl;
  try{
    d.run();
  }
  catch(...){

    std::cout<<"caught exception running events."<<std::endl;

    //
    // Can't really do anything graceful here like continue
    // to handle connected sockets.
    //
    // I could try to call run again, but I doubt that's safe.
    // as we could have been mucking with internal structures
    // when the exception let go.
  }

  std::cout<<"all done"<<std::endl;
  std::cout<<"post_count: "<<post_count
           <<std::endl;
  std::cout<<"call_count: "
           <<call_count<<std::endl;

  return 0;
}

[boost] asio queue OOM stress test

christopher baus