Workaround for using synchronous modules in AnyEvent?

[Marc Lehmann]

On Mon, Apr 15, 2013 at 11:27:25PM -0700, Mike Schilli <anyevent at perlmeister.com> wrote:
> So, I've dabbled around with this a bit and I think the best solution
> is to write AnyEvent::ZooKeeper. Zookeeper offers an asynchronous C API,
> now I'm working on integrating it with an event loop that I can then
> use with AnyEvent.

A formidable task.

> So far so good. However, ZooKeeper client API calls aren't using a traditional
> event loop, but offer two API calls to keep the system ticking:
> 
>     zookeeper_interest(zh, &fd, &interest, &tv);
>     (https://github.com/apache/zookeeper/blob/trunk/src/c/src/cli.c#L741)
> 
> does some preliminary work and returns a number of file descriptors
> to watch for incoming data.

> So I guess what needs to be done to make this work with AnyEvent is: Call
> zookeeper_interest, get the file descriptors, then tell the event loop that
> AnyEvent's currently running that we want to watch for these file descriptors
> and call a callback if something happens. Is this possible in a somewhat
> event loop agnostic way?

This kind of API is not uncommon, but unfortunately, you cannot
generically put this API on top of AnyEvent (and neither on many event
loops, which is why AnyEvent can't do it).

In theory, you could just query the fd set and timeout value and then
create a timer and as many io watchers as needed.

The problem is that this kind of API is made to work with select, and not
with event loops: it doesn't tell you *when* to query the fds.

To make it work generically with event loops, you'd also need an
event-based interface in zookeper, specifically:

1. you need to recreate the timer when the timeout target changes.
2. you need to recreate the I/O watchers when an fd is added or removed.
3. you need to recreate the I/O watchers when an fd is changed (but keeps the
   same value).

Most libraries fail with 1. and 2., and almost all of them fail to
properly report 3., because 3. isn't required when using select.

Emulating select is possible with e.g. EV (because it has prepare/check
watchers), but not with AnyEvent alone.

Maybe zookeeper has an interface that tells you about these changes, in
which case it's simple: each time there is a change, destroy all old
watchers, and then create a timer and I/O watches as needed.

If zookeper doesn't, you could try to analyze zookeeper to see if it still
works. You could, for example, recreate the watchers each time you expect
a change, e.g. when you call a zookeeper function that changes fds or
timeouts. Or maybe zookeeper only ever opens one connection per host,
and never changes it (no reconnect), so one watcher per lifetime of the
connection object might be enough.

This would effectively mean implementing event sources for 1-3 on your
own. And that is usually possible, depending on how much extra effort is
put in, and how stable the implementation is.

A typical failure case for 1. and 2. would be: the timeout changes, but you
still run with the old value, delaying the timeout. Or a fd is closed, but
you still have a watcher for it, because you weren't told about it.

A typical failure case for 3. would be: a connection fails, is closed, a
new connection is made that gets the same fd number, which results in no
change, so you nevr know about it. That works with select, but not with
epoll.

The way to generically implement this with EV would be to create a prepare
watcher that *always* creates timer and I/O watchers, and a check watcher
that checks whether any of these watchers have seen events, and destroy
them again.

Since that creates watchers before every single select/epoll etc. call, it
effectively emulates select, at the cost of creating I/O watchers on every
loop iteration (which isn't so efficient with epoll for example).

> Or would it be smarter to run Coro's select as you pointed out in your
> previous response?

Well, it might probably be more convenient for you, as somebody has
written the "generate watchers and check them in a select-compatible
way" code already, but it would (effectively) make your module some
Coro::Zookeeper module (because most people expect AnyEvent modules not
to have a dependency on Coro). The code isn't very complicated or long
either, you could just look at Coro::Select and do the same in your code.

However, the principial problem stays the same: what happens when the
watchers should change, but your Coro thread is currently executing
Coro::Select::select?

You could either ensure that only the thread doing the select does make
changes, or you could cancel the thread and recreate it, both of which are
not very appealing.

So, if you can solve it with Coro::Select, you can probably solve it with
AnyEvent alone, at little extra effort.

-- 
                The choice of a       Deliantra, the free code+content MORPG
      -----==-     _GNU_              http://www.deliantra.net
      ----==-- _       generation
      ---==---(_)__  __ ____  __      Marc Lehmann
      --==---/ / _ \/ // /\ \/ /      schmorp at schmorp.de
      -=====/_/_//_/\_,_/ /_/\_\