AIO integration

Marc Lehmann schmorp at schmorp.de
Fri May 2 13:19:44 CEST 2008


On Thu, May 01, 2008 at 12:20:36AM +0300, Paul Colomiets <pc at gafol.net> wrote:
> I'm looking for integration of libev with
> asynchronous io for files. It should be
> integrated quite naturally, using
> real-time signals for linux and other
> posix compatible OSes, and probably

real-time signals are neither natural nor imho a good way to do that (the
implementation complexity is enourmous), besides, linux doesn't have
generic asynchronous I/O for files yet (and I don't really see plans to
add it anytime soon, the same for solaris).

> Does anybody tryed to implement
> that?

there is a very portable perl interface for asynchronous I/O called
IO::AIO then I can recommend full-heartedly :) (it uses threads for most
things, but virtualises operations such as stat or open which often are
high latency generators and for which no aio interfaces are available
traditionally).

> I think interface should look like:
> 
> ev::fileio io;
> io.set(fd, ev::READ)
> io.start(buf, buflen)

You forgot the file offset.

I think a mature aio interface needs to cope at least with open and stat
as well (and then probably the full host of POSIX functionality).

> Probably operations can be collected
> for a single loop iteration and started
> with lio_listio if more than one.

Yes, or one could issue them when they happen. I don't see much avdantage
to caching here.

> If no implementation is available, is
> anybody interested in this kind
> of patch for libev? Or would it
> be better as a separate library?

I think it would be best as a separate library at leats initially, as,
unlike libev, there are few precursors, so the api probably needs a longer
design phase (and libev sets limits of what you can do due to it having to
be embeddable).

Feel free to discuss the design here, though. I'd have zero problems also
managing the repository or making it *the* aio package for libev at some
point.

I do think that the posix aio_* functions are not ready yet(!) and that
realtime signals are a dead end.

I think it would be far more sensible to use a traditional threadpool
approach (and code for that already exists, just not in a libev-like way),
and use other, potentially more efficient native aio, only for the things
where it works.

I am also not completely convinced at the moment that supporting windows
fully would make sense, as that would severely limit the API that is
possible (reading/writing isn't an issue, but opening files is, as well as
having an API for posix-like stuff like file attributes).

Of course, your goal might simply be read+write, but I think that would
be too limited to be useful, as the additional value over aio_xxx is not
really there.

(note that libev already supports async-wakeups via ev_async very
efficiently)

-- 
                The choice of a       Deliantra, the free code+content MORPG
      -----==-     _GNU_              http://www.deliantra.net
      ----==-- _       generation
      ---==---(_)__  __ ____  __      Marc Lehmann
      --==---/ / _ \/ // /\ \/ /      pcg at goof.com
      -=====/_/_//_/\_,_/ /_/\_\



More information about the libev mailing list