using libev with multiple threads

[Chris Herssens]

Thanks for the example.

I see that you use a pipe for notifying the worker thread that there is a
new connection.
I don't to this in my code but I use ev_async_send() for notifying the
worker thread ( see attachment files for the worker thread notification)
My implementation is done as follows

1.  each worker thread contains a lock free queue and has an ev_async
watcher to listen on
2. if there arrives a new connection on listen thread, the listen thread
creates an ev_io watcher and puts this on the lock free queue of a worker
thread.
3. the listen thread notifies the worker thread with av_sync_send()
4. the callback of this ev_sync takes all the ev_io watchers from the queue
and start watching on these watcher.

I don't know if this implementation always works, because I'm not sure that
the worker thread catches all async events from the listen thread.
So can it be possible that when the listen thread wants to notify a worker
thread using ev_sync_send, that the callback isn't call ?

Is it possible to avoid this lock free queue ? Is it for instance thread
safe to start the ev_io watcher from the listen thread and then call only
av_sync_send() ?
I.e can I do the follow call from the listen thread
1. ev_io_start(evLoop, watcher), where evLoop is the worker thread event
Loop (I suppose not)
2. ev_async_send(evLoop,asyncWatcher)  (this call is fine ?)


regards


On Sat, Feb 2, 2013 at 12:39 AM, 钱晓明 <mailtoantares at 163.com> wrote:

> I am happy to see this e-mail because I can contribute some thing :-)...
> This is exactly the same need for me and I write some code to do
> this. They are in attachment files.
> The key to accomplish this goal is:
> 1. create new event loop for per worker thread, with a pipe and a extra
> ev_io to watch pipe's read event. This is done before the worker thread
> start, so the main thread(listen thread) can hold these in a struct in a
> list. It is important to know that pipe's read event is watched in the
> worker thread's event loop, not the main event loop.
> 2. once new connection accepted, the listen thread send the accepted
> socket fd on pipe, just using round robin to determine which thread
> struct(this contain the pipe) to use
> 3. the worker thread use a particular callback for the pipe read watcher.
> In this callback(*dispatched_socket_arrived *in example), create normal
> read/write watchers for socket fd just received on pipe. Just start the
> read watcher, not start the write watcher for this socket fd
> 4. so data send/received on this socket fd is processed on worker thread.
> I hope this can help you.
>
> 在2013年01月31 22时31分，"Chris Herssens"<chris.herssens at gmail.com> 写道：
>
>
> I want to use libev with multiple threads for the handling of tcp
> connections. What I want to is:
>
>    1.
>
>    The main thread listen on incoming connections, accept the connections
>    and forward the connection to a workerthread.
>    2.
>
>    I have a pool of workerthreads. The number of threads depends on the
>    number of cpu's. Each worker-thread has an event loop. The worker-thread
>    listen if I can write on the tcp socket or if somethings available for
>    reading.
>
> I looked into the documentation of libev and I known this can be done with
> libev, but I can't find any example how I have to do that.
> Does someone has an example? I think that I have to use the ev_loop_new()
> api, for the worker-threads and for the main thread I have to use the
> ev_default_loop() ?
>
> Since I new in using libev I really need an example how to dispatch an
> incoming tcp connection to a worker-thread. I have started to implement it,
> but I don't get it work. In the main thread I start the default loop. And
> if I received new connection I create a watcher and use the ev_io_start
> with the loop that I create in the workerthread. It seems that the callback
> function of the watcher is not always triggered.  Also Is there a way to
> block the ev_loop if there no active watchers ?
>
>
> regards.
>
>
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.schmorp.de/pipermail/libev/attachments/20130202/753e7911/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: EventThread.cpp
Type: text/x-c++src
Size: 1716 bytes
Desc: not available
URL: <http://lists.schmorp.de/pipermail/libev/attachments/20130202/753e7911/attachment-0001.cpp>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: EventThread.d
Type: application/octet-stream
Size: 44639 bytes
Desc: not available
URL: <http://lists.schmorp.de/pipermail/libev/attachments/20130202/753e7911/attachment-0001.obj>