libev loop in a Python thread

[lekmalek]

On Sun, 06 Jun 2010 22:18:34 +0200
"Jonas H." <jonas at lophus.org> wrote:

> hello there,
> 
> I have written a small standalone Python web server in C using libev.
> I want to run it within a Python thread to make it possible to share 
> memory easily.
are you using pyev or did you roll out you own python wrapper?

> 
> Unfortunately it seems like the thread that runs the libev mainloop 
> blocks any other Python thread:  Once the libev loop has been
> started, code in other threads won't be executed any more.  If I send
> a request to the server -- so the libev thread leaves the libev loop
> code and enters my web server C code -- other Python threads' code is
> executed. Until the C code re-enters the libev code.
yep, what you describe is standard behavior if you didn't release the
GIL in you C/Python code. You need to release it before calling ev_loop
and reacquire it when it returns.
more info:
http://docs.python.org/c-api/init.html#thread-state-and-the-global-interpreter-lock

> 
> I tried threading priorities, but that didn't change anything.
> Changing the backend doesn't change anything, either (I tried epoll
> and select).
> 
> I debugged around a little bit and found out that the other Python 
> threads all stop some mystic `sem_wait`.  Not that this is something
> to worry about -- but could it be that something deadlocks there?  Or
> is it just libev that doesn't give the poor(ly implemented) Python
> interpreter a chance of taking back control? ;-)
> 
> I'm glad about any suggestions on how to fix this stuff.
try the Py_BEGIN_ALLOW_THREADS/Py_END_ALLOW_THREADS macros around the
blocking calls (mainly ev_loop). Note, that you will need to ensure
that your python callbacks are also protected by the GIL.

hope it helps

malek

> 
> Thanks!
> 
>      Jonas
> 
> 
> PS: If you want code:
> 
>    git clone git://github.com/jonashaag/bjoern
>    cd bjoern
>    make get-http-parser
>    make
>    python tests
> 
> _______________________________________________
> libev mailing list
> libev at lists.schmorp.de
> http://lists.schmorp.de/cgi-bin/mailman/listinfo/libev