6. The ideas about I/O filtering as discussed for Apache (see http://www.serverwatch.com/tutorials/article.php/1129721) need to be analyzed and probably implemented in IOStreams before we can really design the demultiplexor library.
This is less trivial I am afraid. But nevertheless an important design issue as you will agree. I think that the author of IOStreams, Jonathan, agrees with me that the ideas of the Apache people concerning how to treat data that is filtered should be handled, is something important that needs support in a future version of IOStreams. This idea basically means that data that is being passed on to filters is 'tagged' with a type that signifies its origin; for example - sockets, memory mapped files, a disk file etc. I think that the way of how we incorporate these 'types' into the data/objects that are passed on will have a significant influence on how we need to design event dispatching for the different types. For example - an application that does both, handle sockets and read and process a very large input file (of a few GB say) might spend so much time on processing the file that it neglects the sockets too much. I already brought up the need for threads for this (see point 2). A 'file dispatcher' thread could for example read only a given amount of data per time unit (limit the bandwidth). In most current implementations there are two approaches: 1) The file is processed (read) untill read(2) would block; likely that approach leads to processing the whole file. 2) The file is read until its buffer is "full" - then the application continues with the main loop which will process all data in the buffer - handle ONE cycle of sockets - and then start to read the file again. This only works when there is a correct balance between the need to handle the sockets and the size of the file buffer. If the buffer would be growable without limits then this certainly won't work. We cannot rely of arbitrary ways of distributing cpu over different sources like this. A more controlled priority management is needed. -- Carlo Wood <carlo@alinoe.com>