Optimal multithread model

Marc Lehmann schmorp at schmorp.de
Mon Mar 15 13:09:23 CET 2010

On Mon, Mar 15, 2010 at 11:48:54AM +0100, Christophe Meessen <christophe at meessen.net> wrote:
> It looks like libev is designed to have one thread calling the ev_loop  
> and executing the callbacks (assumed to be fast and never blocking).
> This is a straightforward design but it is not performance optimal when  
> requests take a very short time to execute and the system is very loaded  

It's probably the most efficient model, actually?

> (many threads/process). This due to passing the request to another  

Keep in mind thatthe primary use for threads is improve context switching
times in single processor situations - event loops are usually far faster at
context switches.

Now that multi cores become increasingly more common and scalability to
multiple cpus and even hosts is becoming more important, threads should be
avoided as they are not efficiently using those configs (again, they are a
single-cpu thing).

While there are exceptions (as always), in the majority of cases you will
not be able to beat event loops, especially whne using multiple processes,
as they use the given resources most efficiently.

> thread for processing. If the system is well loaded, it may take a  
> significant time until the worker thread is given the CPU by the  
> scheduler to process the request.

Yes, threads were not meant for these kinds of workloads, and their
overheads are enourmous even on systems that are very fast (such as

> itself. But some requests may take a long time to execute. So trick is  
> to make the worker threads waiting to be able to monitor. When the  
> monitoring thread detects an event, it signals the worker thread pool,  

That sounds very very slow. When you can't use a more efficient process model
and for some reason *have* to use threads, a lader/follower pattern, or
multiple event loops, are usually more preferable. YMMV.

> This multi thread model reduces request processing latency and supports  
> callbacks with long time time to execute.

It's also typically very inefficient, BUT:

> My understanding is that it would simply require the addition of a  
> ev_monitoring() function which would be called by the worker threads.  

.. maybe I didn't understand your problem, but I don't see why you need
extra functionality, you can just let another thread continue polling for
events - libev supports multiple threads running the same event loop as
long as you lock.

(But to repeat, all this locking is costly and wastes a lot of
concurrency, as threads generally do - forcing threads to work like
independent entities does not work well if you have multiple cpus, as
threads make it impossible to use multiple cpus independently, which
unfortunately is required for performance).

> Some tweaks are required to ensure that there is always at least one  
> thread actively monitoring or ready to take over monitoring. Callbacks  
> should be queued if there is not enough working threads.

Callbacks are already queued inside libev - and "monitoring" in the case
of all threads being busy doesn't seem to be so useful to me (so I am not
sure why it would be required).

> I could not see how this could be implemented with the current libev  
> API. Did I miss something ?

I can only guess you missed the THREADS section?

                The choice of a       Deliantra, the free code+content MORPG
      -----==-     _GNU_              http://www.deliantra.net
      ----==-- _       generation
      ---==---(_)__  __ ____  __      Marc Lehmann
      --==---/ / _ \/ // /\ \/ /      schmorp at schmorp.de
      -=====/_/_//_/\_,_/ /_/\_\

More information about the libev mailing list