Anything wrong with using AnyEvent::Handle for files?

Marc Lehmann schmorp at schmorp.de
Sat Aug 8 05:42:12 CEST 2015


On Fri, Aug 07, 2015 at 08:19:13AM -0700, Mike Schilli <office at perlmeister.com> wrote:
> Although the documentation seems to discourage using AnyEvent::Handle
> for anything but streaming things like sockets and pipes, using it for
> files seems to work fine in the code below.
> Any reason not to use it in this way?

Daniel summarised the technical aspects quite nicely in his reply - I'll
just add some philosophical thoughts and a few more obscure technical
issues to it.

First of all, if you use it, and it works for you, and you won't come to me
to complain if it breaks, feel free to use it on files.

Unlike for packet sockets, where AnyEvent::Handle refuses to work because it
can't work properly with them, AnyEvent::Handle does best effort for files,
for reasons explained later.

However, there are limitations. For example, AnyEvent::Handle doesn't care
for append mode or other file flags - it treats its handles as streaming
files, so only "normal" file handles have good chances of working
reliably. Moreso, AnyEvent::Handle relies on AnyEvent, and AnyEvent
doesn't support files in its I/O watchers (it's indeed documented).

The reason for this is that, although all sane event mechanisms (poll,
select...) do the right thing with nonblocking file handles, as usual,
epoll literally fails with them. This failure case isn't well documented,
so most implementations get it wrong and fail in various ways. For
example, tmux (based on libevent) ran into this problem, as libevent
doesn't handle files when it is using epoll as it's backend.

EV (based on libev) does handle files correctly, and most other event
loops still use selector poll nowadays. For those backends who do use
epoll, I would assume they won't handle files until I have verified that
they do.

Now to something different: AnyEvent::Handle is made to handle event-based
streaming, which means pipes and sockets usually, and often makes little
sense for files indeed. There are, however, exceptions to this:

First, it should be possible to use AnyEvent::Handle on STDIN/STDOUT, and
have it work both for pipes and files. While it might make sense to use
normal perl I/O for files, it makes little sense to write two versions of
your parser, one with AnyEvent::Handle, one with native perl I/O.

Second, AnyEvent::Handle *can* be nicer to parse files than the native perl
I/O. For example, reading concatenated old JSON texts from a file is probably
easier with AnyEvent::Handle, and it's hard to see why one should be forced
to resort to another technique when one already knows how to do it with
AnyEvent.

So, I *do* think that AnyEvent::Handle should support files, but it's not as
easy as changing the documentation. Basically, there are multiple
alternatives:

1. make AnyEvent::Handle detect this case and use e.g. AnyEvent::IO. This
   is not trivial (AnyEvent::IO is ghastly low level :), and of course
   limited to AnyEvent::Handle, but would have the benefit of actually not
   blocking on slow disk or network I/O.

2. guarantee that files work with AnyEvent's I/O watchers and make
   them work. this would be beneficial for people who do their own
   AnyEvent::Handle style stuff, but would be "blocking" on I/O. It might
   incur a rather large performance overhead for some backends though, and
   AnyEvent doesn't control all backends, and was written basically to
   support the common features of existing loops.

3. convince existing backend authors to support files in their I/O watchers.
   AnyEvent->io will then just work, as well as AnyEvent::Handle, at least
   for normal files, regardless of the documentation.

Of course, I'd preferably go for 3), and maybe later 1)...

Right now, however, neither of these are sufficiently high on my TODO list
to lift this limitation soon.

... as a module author, you *could* just use AnyEvent::Handle and blame
any backends that don't get it right, of course.

-- 
                The choice of a       Deliantra, the free code+content MORPG
      -----==-     _GNU_              http://www.deliantra.net
      ----==-- _       generation
      ---==---(_)__  __ ____  __      Marc Lehmann
      --==---/ / _ \/ // /\ \/ /      schmorp at schmorp.de
      -=====/_/_//_/\_,_/ /_/\_\



More information about the anyevent mailing list