Using libev with pthreads and one listening socket
celtic at sairyx.org
Wed Mar 30 06:27:04 CEST 2011
> I'm using one event loop per thread, and so
> > far the best I've been able to come up with is to "round-robin" the
> > ev_io watcher on the listening socket itself :
> (the leader/follower-pattern)
> If you use linux and epoll, then yes, every start and stop will likely lead
> to a syscall in your situation.
> and ev_async will add another two syscalls.
> and on multiple cores, this might become even slower.
This seems to be the case, yes - it's simply not worth the effort in
throwing the leader around.
> Linux unfortunately has no good event interface - select/poll work nicely
> but don't scale, epoll does scale but is otherwise beyond brokenness, and
> has a high overhead.
I enjoyed reading your (many) comments in the documentation talking
about the same :-P, particularly "a frankenpoll" re: epoll.
This will be deployed on FreeBSD, so the performance characteristics may
end up varying anyway.
> Tough situation :)
> However, with luck, your performance issue will evaporate once your proxy
> does actual work, as the extra syscall overhead will then reduce.
This seems to be the case. Right now I'm still employing the (possibly
criminal) method of having all threads listen on the fd - it saves on
the syscalls -- maybe I don't need to worry about that anyway -- but
it's also a simpler design.
Thanks very much for your input!
> On Mon, Mar 14, 2011 at 06:14:00PM +1100, Arlen Cuss <celtic at sairyx.org> wrote:
> > I've partly resolved my own complain; I've actually started instead to
> > watch the listener fd on *all* loops at once. It works on Linux at
> > least, though I've yet to try it on my other target (BSD). I'm trying
> > for portability, so if it doesn't cut it, I'll have to look elsewhere.
> It should *work* everywhere, but performance is another issue. My guess is
> that you get one wakeup in the lucky case of an idle thread accepting the
> connection quickly enough, and an all-wakeup when all your loops are busy.
> Can't imagine that will help with performance, but under very high load,
> this basically degrades to polling once per loop, which is probably much
> more efficient than any leader/follower scheme.
More information about the libev