Feature request: ability to use libeio with multiple event loops

[Hongli Lai]

On Tue, Dec 20, 2011 at 5:06 PM, Marc Lehmann <schmorp at schmorp.de> wrote:
> Threads were meant as an optimisation for single cpu systems though, and
> processes are meant for multiple cpus (or cores), and use the available
> hardware more efficiently.

I would like to know more about this claim. It's not that I don't
believe you, but I'm genuinely interested in this area and I hear all
kinds of (often contradicting) information about threads vs processes.
What is it about processes that are more efficient than threads? For
the sake of simplicity let's limit ourselves to x86 Linux and x86_64
Linux. I don't really care about other platforms.


>> 2. All threads are mostly independent, but share one data structure.
>> It's faster to query and to manipulate this data structure with locks
>> and shared memory than to use message passing and multiple processes.
>
> Even if true (I am a bit doubtful) thats a difference between message
> passing and shared memory, not between processes and threads, Processes
> would still be more efficient overall.

I know it's not a strict difference between processes and threads, but
in my case using processes would pretty much force me to use message
passing. I know that shared memory exists, however inter-process
shared memory locks are either not portable
(pthread_mutexattr_setpshared is not implemented on many platforms) or
slow (file locks; flock() and fcntl() *always* require a system call
to the kernel). So that only leaves:
- sockets and pipes with read()/write()
- SysV message queues (doesn't integrate well with select() so kinda useless)
Both of these methods require:
1. A system call to the kernel for the request message.
2. A context switch to the peer process.
3. Another call to the kernel for the response message.
4. A context switch back to the original process.
While with locks, it's just an atomic instruction in the uncontended
case; no kernel intervention, message (de)serialization or context
switches needed.


> Whether context switching flushes the TLB depends on the architecture,
> not so much on Linux (which presumably suppreses it if unnecessary). Note
> that threads incur extra overheads on multiple cpu cores as well, where
> processes are more efficient.

More efficient in what way?


>> Manipulating a shared data structure through message passing *always*
>> involves a context switch, even in the uncontended case, while mutexes
>> do not impose such overhead in the uncontended case.
>
> Thats not true either, where did you pick this up?

See above. With all due respect, I'm actually wondering why you don't
think what I said is true.
It's true that *conceptual* message passing necessarily doesn't
require kernel system calls or context switches, but I'm talking about
OS-level message passing and OS-level processes.

- Hongli
-- 
Phusion | Ruby & Rails deployment, scaling and tuning solutions

Web: http://www.phusion.nl/
E-mail: info at phusion.nl
Chamber of commerce no: 08173483 (The Netherlands)