Anything wrong with using AnyEvent::Handle for files?

Mike Schilli office at perlmeister.com
Tue Aug 11 04:15:04 CEST 2015


> First of all, if you use it, and it works for you, and you won't come to me
> to complain if it breaks, feel free to use it on files.

Funny you should say that, as I found indeed something wrong with it.
I've moved away from doing it that way based on everybody's
recommendations here, but just for your personal amusement: It seems
that $fh isn't advanced, so "tell $fh" stays at 0 and subsequent
checks (e.g. for meanwhile appended data) fail:

$ cat foo.txt
one
two
three

$ cat testme
#!/usr/local/bin/perl -w
use strict;

use AnyEvent;
use AnyEvent::Handle;

open my $fh, "<", "foo.txt" or die;

my $handle = AnyEvent::Handle->new(
    fh       => $fh,
    on_error => sub { die "read error: $!"; },
    on_eof   => sub { print "done!\n"; },
    on_read  => sub {
        my( $hdl ) = @_;

        $hdl->push_read( line => sub {
            my( $h, $line ) = @_;
            print "line=$line\n";
            print "fhtell=", tell $fh, "\n";
        } );
    },
);

AnyEvent->condvar->recv;

$ ./testme
line=one
fhtell=0
line=two
fhtell=0
line=three
fhtell=0
done!

On Fri, Aug 7, 2015 at 8:42 PM, Marc Lehmann <schmorp at schmorp.de> wrote:
> On Fri, Aug 07, 2015 at 08:19:13AM -0700, Mike Schilli <office at perlmeister.com> wrote:
>> Although the documentation seems to discourage using AnyEvent::Handle
>> for anything but streaming things like sockets and pipes, using it for
>> files seems to work fine in the code below.
>> Any reason not to use it in this way?
>
> Daniel summarised the technical aspects quite nicely in his reply - I'll
> just add some philosophical thoughts and a few more obscure technical
> issues to it.
>
> First of all, if you use it, and it works for you, and you won't come to me
> to complain if it breaks, feel free to use it on files.
>
> Unlike for packet sockets, where AnyEvent::Handle refuses to work because it
> can't work properly with them, AnyEvent::Handle does best effort for files,
> for reasons explained later.
>
> However, there are limitations. For example, AnyEvent::Handle doesn't care
> for append mode or other file flags - it treats its handles as streaming
> files, so only "normal" file handles have good chances of working
> reliably. Moreso, AnyEvent::Handle relies on AnyEvent, and AnyEvent
> doesn't support files in its I/O watchers (it's indeed documented).
>
> The reason for this is that, although all sane event mechanisms (poll,
> select...) do the right thing with nonblocking file handles, as usual,
> epoll literally fails with them. This failure case isn't well documented,
> so most implementations get it wrong and fail in various ways. For
> example, tmux (based on libevent) ran into this problem, as libevent
> doesn't handle files when it is using epoll as it's backend.
>
> EV (based on libev) does handle files correctly, and most other event
> loops still use selector poll nowadays. For those backends who do use
> epoll, I would assume they won't handle files until I have verified that
> they do.
>
> Now to something different: AnyEvent::Handle is made to handle event-based
> streaming, which means pipes and sockets usually, and often makes little
> sense for files indeed. There are, however, exceptions to this:
>
> First, it should be possible to use AnyEvent::Handle on STDIN/STDOUT, and
> have it work both for pipes and files. While it might make sense to use
> normal perl I/O for files, it makes little sense to write two versions of
> your parser, one with AnyEvent::Handle, one with native perl I/O.
>
> Second, AnyEvent::Handle *can* be nicer to parse files than the native perl
> I/O. For example, reading concatenated old JSON texts from a file is probably
> easier with AnyEvent::Handle, and it's hard to see why one should be forced
> to resort to another technique when one already knows how to do it with
> AnyEvent.
>
> So, I *do* think that AnyEvent::Handle should support files, but it's not as
> easy as changing the documentation. Basically, there are multiple
> alternatives:
>
> 1. make AnyEvent::Handle detect this case and use e.g. AnyEvent::IO. This
>    is not trivial (AnyEvent::IO is ghastly low level :), and of course
>    limited to AnyEvent::Handle, but would have the benefit of actually not
>    blocking on slow disk or network I/O.
>
> 2. guarantee that files work with AnyEvent's I/O watchers and make
>    them work. this would be beneficial for people who do their own
>    AnyEvent::Handle style stuff, but would be "blocking" on I/O. It might
>    incur a rather large performance overhead for some backends though, and
>    AnyEvent doesn't control all backends, and was written basically to
>    support the common features of existing loops.
>
> 3. convince existing backend authors to support files in their I/O watchers.
>    AnyEvent->io will then just work, as well as AnyEvent::Handle, at least
>    for normal files, regardless of the documentation.
>
> Of course, I'd preferably go for 3), and maybe later 1)...
>
> Right now, however, neither of these are sufficiently high on my TODO list
> to lift this limitation soon.
>
> ... as a module author, you *could* just use AnyEvent::Handle and blame
> any backends that don't get it right, of course.
>
> --
>                 The choice of a       Deliantra, the free code+content MORPG
>       -----==-     _GNU_              http://www.deliantra.net
>       ----==-- _       generation
>       ---==---(_)__  __ ____  __      Marc Lehmann
>       --==---/ / _ \/ // /\ \/ /      schmorp at schmorp.de
>       -=====/_/_//_/\_,_/ /_/\_\



More information about the anyevent mailing list