Optimal multithread model

Marc Lehmann schmorp at schmorp.de
Wed Mar 17 02:04:36 CET 2010

On Tue, Mar 16, 2010 at 05:32:42PM -0500, Dan White <dwhite at olp.net> wrote:
>> userspace thread switching is 10-100 times faster than kernel thread
>> switching on my GNU/Linux box - and processes don't need to communicate
>> with themselves via slow main memory (they can use cache).
> Marc,
> Can you provide more information on userspace thread switching?

If you have specific questions, I might try to answer.

The above is my personal experience from many benchmarks - for example, I
can context switch perl "Coro" threads about 40 times as fast when using
userspace context switching as opposed to using real threads. This number
can be dramatically higher when you pin all threads on a single core (or
lower), which depends very much on the usage pattern.

I could measure the same with libcoro (a c library for portable userspace
context switching) on netbsd: with "old" netbsd, I got around 2-3 mio
switches/s, with "new" netbsd (which has kernel threads) the number has
sunk to 80k/s on the same hardware.

Which makes total sense, as basiclaly, in userspace a context switch is 10
instructions in it's most basic form (save callee-saved registers, swap
stakc pointer, restore registers), while the kernel has to jump through
many hoops - event he most simple syscall on linux takes 300+ cycles, a
context switch can be many times as expensive.

> Do you have any good links that explain this model?

Not really, no. It's basically just doing some kind of multithreading
in userspace instead, which isn't very limiting - everybody does it

Apart from libcoro (extremely bare bones, unlikely to make people happy,
think libev to get the idea :) there is also GNU libpth, and various
so-called "green threads" implementations.

Kernel threads are, however, replacing userspace threading almost everywhere
at tremendous speed.

This is, in my opinion, not a good thing - kernel threads are IMHO a must
have, but so are userspace threads.

Compare that with perl's native threads (which actually emulate processes,
not threads - it's basically windows fork emulation for unix). Those
"threads" are enourmous memory and cpu hogs, make everything much slower
even when not used, btu almost every GNU/Linux distribution has them -
because they use pthreads and "thread" is a killer buzzword.

As with anything, there is no golden way that solves all your problems.
Kernel threads are important and useful, but they are not the only or best
way to solve every problem.

Or to quite Rob Pike:

   Threads are like salt.  I like salt, you like salt, we all
   like salt, but we eat more pasta than salt.

                The choice of a       Deliantra, the free code+content MORPG
      -----==-     _GNU_              http://www.deliantra.net
      ----==-- _       generation
      ---==---(_)__  __ ____  __      Marc Lehmann
      --==---/ / _ \/ // /\ \/ /      schmorp at schmorp.de
      -=====/_/_//_/\_,_/ /_/\_\

More information about the libev mailing list