libev-4.31 has just been released

[Marc Lehmann]

On Sat, Dec 21, 2019 at 06:45:20PM +0000, Benjamin Mahler <benjamin.mahler at gmail.com> wrote:
> Sounds like some of the iouring findings are surprising to Jens (the
> author).

Well, I mailed him personally (no response), opened bug reports on
bugzilla.kernel.org (no response), and even found a discussion on the most
pressing problem on the linux-kernel list that I found after debugging
this in the kernel, from where I paraphrased linus's reaction.

https://bugzilla.kernel.org/show_bug.cgi?id=204081 that bug
https://bugzilla.kernel.org/show_bug.cgi?id=204065 oops bug
https://bugzilla.kernel.org/show_bug.cgi?id=204085 linux aio potential security bug (not iouring)

https://lore.kernel.org/patchwork/patch/1047453/ the problem in linux aio, same in io_uring
"I guess nobody really expects to use aio_poll on anything but a socket or something?"

I understand we are all busy, and bugzilla.kernel.org sems to work a lot
like the usenet wizard groups (saome areas are just blackholes, such as
btrfs and aio/iouring, others are quite active, such as ext4 :), but on
the other hand, Jens really should not be that _surprised_ :)

Now, if this is what gets his attention, just the better, a working and
performant io_uring could finally get us out of epoll hell, although
kqueue still clearly leads the design, despite its cumbersome API.

In any case, I put most of my findings into the comments at the top of
ther source files - in my panic in trying to rescue what is clearly the
best event api in linux so far, I even started to embed epoll to *maybe*
get it into a working state, which kind of gets us the worst of all
worlds:

http://cvs.schmorp.de/libev/ev_iouring.c
http://cvs.schmorp.de/libev/ev_linuxaio.c

> Is there a benchmark he can run to look into this?

Haven't tried it, but the libevent/libev behcmark should show the bad
performance compared to epoll (and even poll/select):

http://data.plan9.de/bench.c

Can be compiled in a libev directory:

  gcc -O3 -DNATIVE=1 -DEV_MULTIPLICITY=0 -o bench bench.c -I. ev.c event.c -lm

And then, for e.g. 10000 fds:

  LIBEV_FLAGS=1   ./bench -n 10000 -a 100   # select
  LIBEV_FLAGS=4   ./bench -n 10000 -a 100   # epoll
  LIBEV_FLAGS=128 ./bench -n 10000 -a 100   # iouring

I don't actually remember what exactly I benchmarked (it was a number of
throwaway oneliners), but between kernel oopses, having to use epoll, the
issues with io_uring under high load, the abysmal state of documentation
and so on, I didn't really bother to play with it much more in its current
state.

> Do you have more explanation about "silently ignore parts of the requested
> events on an undocumented subset of file description types"?

I hope all required info is in the bugzilla bug report.

If anybody wants more information, I will happily provide more - it's just
that I already made a very large investment just to be disappointed, so I'd
like to have a perspective on whether its really worth it to invest more time
in this backend.

-- 
                The choice of a       Deliantra, the free code+content MORPG
      -----==-     _GNU_              http://www.deliantra.net
      ----==-- _       generation
      ---==---(_)__  __ ____  __      Marc Lehmann
      --==---/ / _ \/ // /\ \/ /      schmorp at schmorp.de
      -=====/_/_//_/\_,_/ /_/\_\