Anything wrong with using AnyEvent::Handle for files?

Daniel Carosone daniel.carosone at gmail.com
Tue Aug 11 05:06:15 CEST 2015


Yes, Handle uses sysread/syswrite, and these bypass the perl buffering that
tell looks at.  It implements its own buffer management; no point doing
both.
Secondly, even if it used read and write instead, tell would still be
wrong, because it would tell the position that Handle's internal buffer had
read (in your example, at eof) rather than the point where the push_read is
at.

I don't see a way to get at this info, but perhaps you could use a custom
read type to keep track of the byte count.

On 11 August 2015 at 12:15, Mike Schilli <office at perlmeister.com> wrote:

> > First of all, if you use it, and it works for you, and you won't come to
> me
> > to complain if it breaks, feel free to use it on files.
>
> Funny you should say that, as I found indeed something wrong with it.
> I've moved away from doing it that way based on everybody's
> recommendations here, but just for your personal amusement: It seems
> that $fh isn't advanced, so "tell $fh" stays at 0 and subsequent
> checks (e.g. for meanwhile appended data) fail:
>
> $ cat foo.txt
> one
> two
> three
>
> $ cat testme
> #!/usr/local/bin/perl -w
> use strict;
>
> use AnyEvent;
> use AnyEvent::Handle;
>
> open my $fh, "<", "foo.txt" or die;
>
> my $handle = AnyEvent::Handle->new(
>     fh       => $fh,
>     on_error => sub { die "read error: $!"; },
>     on_eof   => sub { print "done!\n"; },
>     on_read  => sub {
>         my( $hdl ) = @_;
>
>         $hdl->push_read( line => sub {
>             my( $h, $line ) = @_;
>             print "line=$line\n";
>             print "fhtell=", tell $fh, "\n";
>         } );
>     },
> );
>
> AnyEvent->condvar->recv;
>
> $ ./testme
> line=one
> fhtell=0
> line=two
> fhtell=0
> line=three
> fhtell=0
> done!
>
> On Fri, Aug 7, 2015 at 8:42 PM, Marc Lehmann <schmorp at schmorp.de> wrote:
> > On Fri, Aug 07, 2015 at 08:19:13AM -0700, Mike Schilli <
> office at perlmeister.com> wrote:
> >> Although the documentation seems to discourage using AnyEvent::Handle
> >> for anything but streaming things like sockets and pipes, using it for
> >> files seems to work fine in the code below.
> >> Any reason not to use it in this way?
> >
> > Daniel summarised the technical aspects quite nicely in his reply - I'll
> > just add some philosophical thoughts and a few more obscure technical
> > issues to it.
> >
> > First of all, if you use it, and it works for you, and you won't come to
> me
> > to complain if it breaks, feel free to use it on files.
> >
> > Unlike for packet sockets, where AnyEvent::Handle refuses to work
> because it
> > can't work properly with them, AnyEvent::Handle does best effort for
> files,
> > for reasons explained later.
> >
> > However, there are limitations. For example, AnyEvent::Handle doesn't
> care
> > for append mode or other file flags - it treats its handles as streaming
> > files, so only "normal" file handles have good chances of working
> > reliably. Moreso, AnyEvent::Handle relies on AnyEvent, and AnyEvent
> > doesn't support files in its I/O watchers (it's indeed documented).
> >
> > The reason for this is that, although all sane event mechanisms (poll,
> > select...) do the right thing with nonblocking file handles, as usual,
> > epoll literally fails with them. This failure case isn't well documented,
> > so most implementations get it wrong and fail in various ways. For
> > example, tmux (based on libevent) ran into this problem, as libevent
> > doesn't handle files when it is using epoll as it's backend.
> >
> > EV (based on libev) does handle files correctly, and most other event
> > loops still use selector poll nowadays. For those backends who do use
> > epoll, I would assume they won't handle files until I have verified that
> > they do.
> >
> > Now to something different: AnyEvent::Handle is made to handle
> event-based
> > streaming, which means pipes and sockets usually, and often makes little
> > sense for files indeed. There are, however, exceptions to this:
> >
> > First, it should be possible to use AnyEvent::Handle on STDIN/STDOUT, and
> > have it work both for pipes and files. While it might make sense to use
> > normal perl I/O for files, it makes little sense to write two versions of
> > your parser, one with AnyEvent::Handle, one with native perl I/O.
> >
> > Second, AnyEvent::Handle *can* be nicer to parse files than the native
> perl
> > I/O. For example, reading concatenated old JSON texts from a file is
> probably
> > easier with AnyEvent::Handle, and it's hard to see why one should be
> forced
> > to resort to another technique when one already knows how to do it with
> > AnyEvent.
> >
> > So, I *do* think that AnyEvent::Handle should support files, but it's
> not as
> > easy as changing the documentation. Basically, there are multiple
> > alternatives:
> >
> > 1. make AnyEvent::Handle detect this case and use e.g. AnyEvent::IO. This
> >    is not trivial (AnyEvent::IO is ghastly low level :), and of course
> >    limited to AnyEvent::Handle, but would have the benefit of actually
> not
> >    blocking on slow disk or network I/O.
> >
> > 2. guarantee that files work with AnyEvent's I/O watchers and make
> >    them work. this would be beneficial for people who do their own
> >    AnyEvent::Handle style stuff, but would be "blocking" on I/O. It might
> >    incur a rather large performance overhead for some backends though,
> and
> >    AnyEvent doesn't control all backends, and was written basically to
> >    support the common features of existing loops.
> >
> > 3. convince existing backend authors to support files in their I/O
> watchers.
> >    AnyEvent->io will then just work, as well as AnyEvent::Handle, at
> least
> >    for normal files, regardless of the documentation.
> >
> > Of course, I'd preferably go for 3), and maybe later 1)...
> >
> > Right now, however, neither of these are sufficiently high on my TODO
> list
> > to lift this limitation soon.
> >
> > ... as a module author, you *could* just use AnyEvent::Handle and blame
> > any backends that don't get it right, of course.
> >
> > --
> >                 The choice of a       Deliantra, the free code+content
> MORPG
> >       -----==-     _GNU_              http://www.deliantra.net
> >       ----==-- _       generation
> >       ---==---(_)__  __ ____  __      Marc Lehmann
> >       --==---/ / _ \/ // /\ \/ /      schmorp at schmorp.de
> >       -=====/_/_//_/\_,_/ /_/\_\
>
> _______________________________________________
> anyevent mailing list
> anyevent at lists.schmorp.de
> http://lists.schmorp.de/mailman/listinfo/anyevent
>



-- 
Dan.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.schmorp.de/pipermail/anyevent/attachments/20150811/23c6f22d/attachment.html>


More information about the anyevent mailing list