AnyEvent recursive blocking wait with Plack

Wed Mar 7 21:18:23 CET 2012

On Tue, Mar 06, 2012 at 04:59:45PM -0800, Mike Schilli <anyevent at perlmeister.com> wrote:
> I'm hitting the 'recursive blocking wait' limitation in AnyEvent and
> wonder how to work around it.

It's less of a limitation in AnyEvent, but more a logical problem that is
so hard to understand (and avoid) that AnyEvent tries to tip you off early
enough.

> In order to make my application portable across web servers, I'm using
> Plack, here's my sample script:
> This works well, but Plack requires that the 'app handler' *returns*

The short version: you can't do that. Think about this: what happens when you
wait there, and another request comes in. Then you recurse again. And
probably again. And probably again.

And if Plack insists on getting a return value then it's not
event-oriented, and chances that this might corrupt it's data structure
are high, because it was obviously not designed for concurrency in this
situation.

> a data structure, whereas AnyEvent typically provides *callbacks*. This

AnyEvent is event-based, Plack's API is blocking - this doesn't go
together.

> that's using callbacks, because then all of a sudden, you can't simply
> "return" data from the main function, because you're in a callback when
> the data is ready.

Welcome to the pains of event based programming, when not everything
supports it:-)

> Any ideas how to work around it? The problem seems to be that Plack
> requires a return value and can't work with a callback. Or does it?

The only way to fix that is to make plack event-based - plack needs to
support some kind of callback (or some kind of continuation object, as
AnyEvent often doesm e.g. in tcp_connect).

Don't call us, we'll call you, is what Plack needs to implement.

For example the AnyEvent::HTTPD module is written with events in mind, and
there, a request(-handler) looks like this:

  $httpd->reg_cb (
     '/' => sub {
        my ($httpd, $req) = @_;

        $req->respond ({ content => ['text/html',
           "<html><body><h1>Hello World!</h1>"
           . "<a href=\"/test\">another test page</a>"
           . "</body></html>"
        ]});
     },

So the server invokes a callback - the request event- and the callback
eventually invokes the response callback on the continuation object ($req)
- the response event.

The only workable alternative is to implement the callback as plack wants,
blockingly, i.e. without AnyEvent,

On Wed, Mar 07, 2012 at 08:01:33AM -0700, Darin McBride <darin.mcbride at shaw.ca> wrote:
> On Tuesday March 6 2012 4:59:45 PM Mike Schilli wrote:
> > This works well, but Plack requires that the 'app handler' *returns*
> > a data structure, whereas AnyEvent typically provides *callbacks*. This
> 
> This is where Coro comes in.  It turns events inside out, allowing you to 
> return the results of a callback.

Well thought, but things are not that simple, unfortunately.

Coro can indeed lift the recursion thing, because you no longer have to
recurse, but it doesn't make Plack event-based out of a sudden.

So, Plack still must support event-based request handling, and most
importantly, the concurrency involved with it. If Plack doesn't support
this kind of concurrency, then you can rely on the event loop.

The only event loops that I know of that support this kind of recursion
are AnyEvent::Loop and EV, and it's tricky to get right.

The only other solution is to use a modal event loop, i.e. create a new
one for the subrequest, so the main event loop doesn't run. AnyEvent does
not support that (there is simply no interface for that, and msot event
loops don't support that either).

So, one can *try* to run Plack inside a coro thread, and hope that the
event loop somehow magically works (not so likely) and that Plack supports
concurrency, but it's not trivial, and hard to debug problems might occur.

Or it might just work, but proving that is hard, even when it seems to
work during testing.

-- 
                The choice of a       Deliantra, the free code+content MORPG
      -----==-     _GNU_              http://www.deliantra.net
      ----==-- _       generation
      ---==---(_)__  __ ____  __      Marc Lehmann
      --==---/ / _ \/ // /\ \/ /      schmorp at schmorp.de
      -=====/_/_//_/\_,_/ /_/\_\