State of IOCP work

[Marc Lehmann]

On Thu, Dec 09, 2010 at 01:23:44PM +0100, Bert Belder <H.Belder at student.TUDelft.NL> wrote:
> > > edge-triggered.
> > 
> > Exactly (but that's not the only problem you also have to generate
> > events for connects and accepts - although this could probably be 
> > handled by doing handle type detection in some way, and some quite 
> > horrible code - at least, thatw as my impression :).
> 
> Why would FD_CONNECT and FD_ACCEPT not work?

Uhm, you are the only one who claims that, so why ask me? In my opinion,
you could probably make ti work with more overhead.

> > Yeah, but since we are callign select anyway, what again is the
> > advantage
> > of select+extra high overhead stuff vs. just select?
> 
> The advantage would be that wait() can wait for other stuff. I don't

Yeah, but youc an already dot hat using the typical windows way, using a
thread (which is *also* required fro your solution btw., with any nontrivial
amount of sockets).

Since libev already handles your case, our original proposition was to make
it handle it faster, but are you sure that having nsockets/63 threads is
really that much faster? Or do you think that writablity is such an uncommon
thign to wait for that it works out in practise?

These are not rhethoric questions, the anwre could actually be "yes" in both
cases, but it's not obvious.

I am not trying to discourage you - if you can get a working backend that
does all the things, then one could look into it and see if it's actually
faster.

My suspicion, though, is that that backend will be more complex than the
rest of libev, and then might not even be that much faster.

> That doesn't mean its difficult to wait for a child process: windows
> will signal the child process handle as soon as the child process exits.
> But select can't wait on that handle.

Yeah, windows is really a rotten design deep by deep. It simply doesn't
have a single event mechanism, but only layers of layers of not
fully-compatible different event mechanisms.

> can wait for a console handle, but not with select. So my idea was to
> add a windows-only ev_winhandle watcher type to handle that kind of
> stuff, but then I need a libev backend that does wait for handles while
> not regressing over the existing select backend otherwise.

Well, you can always start an extra thread that does that, as a first
approximation - the interface doesn't require the overly complex solution you
have in mind (which, if faster, might still be preferable, but the issues
seem to be entirely independent of each other).

This is why there is ev_async_send, and maybe the real problem is that
under some environments, these don't work for whatever reason.

> Wrt the performance impact. Select is particularly slow when you try to
> select on a lot of handles.

Not in my benchmarks actually - some older select() impelmentations did
run a thread for every 63 handles, but newer ones don't seem to do that,
and their speed seems to be linear in the number of sockets (there is
extra overhead in libev, because it needs a slow handle => fd mapping, but
again that could be optimised if anybody cared).

> That wouldn't be needed in this case, because now we'd need to select
> only on sockets for which FD_WRITE is disarmed.

Well, have you tried it out and made sure it really works reliably? :)

> > In theory, it would be possible to make a non-default backend that
> > has additional requirements (as kqueue basically is on many operating
> > systems), such as signalling write result codes or somesuch,
> 
> It could be an alternative to the select trick, if that's what you mean.

We certainly want to keep the select backend - it works stable and is
comparatively simple (having workarounds for the accept bugs on windows
for example).

> It is known that, as long as a socket op doesn't fail with
> WSAEWOULDBLOCK it is writable. And if it does FD_WRITE is rearmed.

Sure, but you don't have that information in libev.

> > but since performance on windows is rather bad even with 
> > "perfect" use of iocps and other advanced features, it's hardly 
> > worth the effort - windows support is good to have, but obviously, 
> > nobody is interested in performance when using windows.
> 
> Sure. Performance isn't my primary concern here, although I wouldn't
> accept a big performance regression over the select backend proper.

If performance is not actually your goal, then the only problem is
that write() fails under mingw, for whatever reason, and this can be
workarounded by using send/recv, which can easily be arranged.

-- 
                The choice of a       Deliantra, the free code+content MORPG
      -----==-     _GNU_              http://www.deliantra.net
      ----==-- _       generation
      ---==---(_)__  __ ____  __      Marc Lehmann
      --==---/ / _ \/ // /\ \/ /      schmorp at schmorp.de
      -=====/_/_//_/\_,_/ /_/\_\