AnyEvent application with Perl backend can enter a loop with 100% CPU usage if the select syscall returns with an error

[Peter Juhasz]

Hi,

we've noticed that one of our AnyEvent applications (a Twiggy server
actually) became unresponsive with 100% CPU usage. Strace has shown
that it kept spamming a select syscall, which returned EBADF. One of
the FDs it was supposed to monitor did not exist anymore.

The problem is that when select returns an error like that, it does so
without delay, and apparently the pure Perl event loop implementation
doesn't handle this error case, hence the 100% CPU usage. Looking at
the source of the module, indeed there is a check for some obsure
Windows case, but not for a generic error.

I'm not quite sure I understand the chain of events that led to the
disappearance of the file handle, but it's reproducible:
Our application runs inside a Docker container, and the sysadmins have
modified something on the host's DNS settings. Some time (about a hour)
later the application failed in the way described above.

I'm not sure what AnyEvent could do in case of such an error, it's
possible that the application is not salvageable at that point. But at
least the error condition could be detected.

best regards,
Peter Juhasz