Optimal multithread model

Marc Lehmann schmorp at schmorp.de
Tue Mar 16 02:23:38 CET 2010

On Mon, Mar 15, 2010 at 11:07:53PM +0000, James Mansion <james at mansionfamily.plus.com> wrote:
>> Keep in mind thatthe primary use for threads is improve context switching
>> times in single processor situations - event loops are usually far faster at
>> context switches.
> No, I don't think so. (re: improve context switching times in single  

Well, that's what they were invented for, whether you think so or not :)

The precise problem they were designed to solve is slow context switching
times due to reloading of cpu registers.

This is a good thing on a single core, having to maintain registers in
multiple cpus concurrently is a problem with multiple cores, something
that processes don't suffer from.

> But you're limited to one core.

With threads, often you are when you want performance - the
synchronisation between cpus when using threads can be very costly, and is
usually unnecessary (why not use the extra hardware you got?).

>> Now that multi cores become increasingly more common and scalability to
>> multiple cpus and even hosts is becoming more important, threads should be
>> avoided as they are not efficiently using those configs (again, they are a
>> single-cpu thing).
> You keep saying this, but that doesn't make it true.

Use your brain then, these things are not so difficult to understand, you
can look them up - start with the history of threads.

Threads force memory/cpu state synchronisation among multiple cores (when
they actually run on multiple cores), which often requires cache flushes, tlb
flushes, synchronous IPIs and so on.

Processes were designed for parallel processing - they don't force these
kinds of synchronisation between cpus, and naturally scale even over host
boundaries, making them both more efficient and more scalable in general.

>> While there are exceptions (as always), in the majority of cases you will
>> not be able to beat event loops, especially whne using multiple processes,
>> as they use the given resources most efficiently.
> Only if you don't block, which is frequently hard to ensure if you are
> using third-party libraries for database access (or heavy crypto, or

That of course is a limitation in those APIs - you don't argue that
threads are more efficient in general because some APIs force you to use
them? That wouldn't make sense.

>> Yes, threads were not meant for these kinds of workloads, and their
>> overheads are enourmous even on systems that are very fast (such as
>> GNU/Linux).

> That is not necessarily so: a spinlock-protected FIFO is hardly an
> enormous overhead, and in lightly loaded systems the added latency
> of passing control is neither here nor there in most cases, while an

userspace thread switching is 10-100 times faster than kernel thread
switching on my GNU/Linux box - and processes don't need to communicate
with themselves via slow main memory (they can use cache).

... and in lightly loaded systems it doesn't matter, but we are talking
about efficiency here, not about how efficiency doesn't matter :)

> ability to spread load over multiple CPU cores is useful. The service

You keep repeating it, but it doesn't make it true :-) Processes spread
their load evenly over cpus, threads cannot by design.

> I'm not discounting multi-process architectures - but sometimes the
> marshalling necessary to flatten data through shared memory (or pipe
> it through the kernel) is painful and the added sharing is handy.  Use
> a modern heap implementation if malloc/free is a bottleneck.

Keep in mind that if a malloc or a free often result in costly IPIs,
copying between CPUs and so on.

Also keep in mind that threads share all memory by default, even when it
makes little sense, performance-wise.

Of course threads can be convinient (although many people would disagree,
as thread programming is very difficult), and inter-process communication
doesn't have so nice standardised APIs, but again, we are talking about
efficiency here, not about how to slow down everything so it is convinient
(the C language is a bad choice for this...).

> a received network request. But writing them off bluntly the way
> you have is crazy.

I have never written off threads "bluntly", and thanks, no, I am not
crazy, I am just interested in architecture, performance and benchmarks.

Not sure what to think of you, I explicitly and often said that threads
are not generally slower. That you think you need to insult me by claiming
I said things I never did is not nice.

Get to know your cache and your timing functions, it might be interetsing

>> threads make it impossible to use multiple cpus independently, which
>> unfortunately is required for performance).
> And the justification for this is what, precisely?

The justification is that these disadvantages make threads good for
single-cpu usage, which is exactly what they were designed for, which IMHO
was a good trade-off. Since you don't know this, it is hardly surprising
that you don't understand their disadvantages.

If you program with threads you should by now be aware of the fact that
threads share a lot of state with each other that requires synchronisation
between all cpus the process.

> There are so many counterexamples to this its hard to avoid them.

Counterexamples disprove the general case how?

> bottleneck on contended state,

Such as shared memory... Think CPU and hardware, not just your APIs. Think
how costly it is to copy memory between cpus, or interrupt another cpu to
update it's TLB cache (or one of the many kernel caches).

This wasn't so much an issue with early SMP systems, but it becomes
increasingly more important.

> but they'd bottleneck if they were processes
> accessing state in shared memory too,

Sure, but processes NEVER share as much as threads do, so the bottleneck
naturally will be smaller.

> and the inevitable sharing of the heap
> data structures isn't nearly the problem it used to be. Even then, often the

It's actually getting worse and worse - one reason is that the memory
hierarchy is acquiring ever more latency these days.

> (Yeah, I know its pointless arguing with Marc, but sometimes you just have
> to say: Marc is talking b*llocks)

I don't think I deserve your treatment - I am trying to help, and I gave an
unbiased overview.

The fact that you a) don't even know the very basics of threads b) didn't
bother to read my e-mail and c) think you need to behave like an asshole
means I will not waste time with you further. This list is for discussing all
things libev, not to promote your thread fanboyism.

Learn to argue with your brain, not your balls.

                The choice of a       Deliantra, the free code+content MORPG
      -----==-     _GNU_              http://www.deliantra.net
      ----==-- _       generation
      ---==---(_)__  __ ____  __      Marc Lehmann
      --==---/ / _ \/ // /\ \/ /      schmorp at schmorp.de
      -=====/_/_//_/\_,_/ /_/\_\

More information about the libev mailing list