When I first added epoll support it was to meet the requirements of handling tens of thousands of connections. I used the level-triggered interface because it mapped easily from the existing select_reactor implementation.
However, I have already been thinking about converting to use edge-triggered epoll to reduce the number of epoll_ctl calls. Some recent changes I have made post-review-version are steps in this direction.
Ok good. I'm glad somebody pointed this out because it was one of the three concerns I was going to mention in my review. If I can't finish my review in the next day, I feel that they've all been addressed else where. Looking at Jody numbers on select() vs epoll(), I realized this is probably the reason why epoll looked bad when there were many "always ready" FDs. There is no reason to wait on an FD until EWOULDBLOCK is returned from read() or write(). Just associate the readiness state with the socket.