[ANN] x0 HTTP server and framework (initial release)

common at gmx.ch common at gmx.ch
Sat Apr 3 16:28:39 CEST 2010

Marc Lehmann wrote:
> On Thu, Apr 01, 2010 at 10:53:39AM +0200, Graham Leggett <minfrin at sharp.fm> wrote:
>> You might be tempted to keep two socket watchers going for listening to 
>> socket read events, and socket write events, but to support SSL  
> This is the mess how it is commonly implemented, and it is indeed horrors.

I've had the non-blocking ssl challenge too, and I use two socket 
watchers, I know it can be done using a single watcher, but ... I'd call 
it a tradeoff, using two io watchers simplified the code and therefore I 
decided to take the performance impact into account.

In fact, given the possibility, I use two io watchers for every type of 
connection, even tcp and udp, and a multitude of timers for each 
connection type to provide timeouts for all kinds of things I want to be 
able to deal with (idle, sustain, listen, throtte_(in,out), connecting, 
dnsresolve, close, handshake, reconnect).

This makes a total of
  * 2 io watchers
  * 10 timers
-> 12 watchers per connection

where 1 io watcher (in) and 2 timers (idle&sustain) are active most of 
the time.

I'm totally aware could be done with 2 watchers, one for io, and one 
timeout and some internal list to have only the 'next' timeout active in 
libev, but I don't feel bad about it.

In fact, libev performs that great, I did not even notice a when it had 
about 6000 connections.

So, if you don't need the extra tidbit of performance, you can keep 
things simple.

> A much easier way is to decouple openssl from the socket altogether,
> i.e. don't let it do socket I/O, instead let it do I/O from/two two
> BIO buffers, which simplifies things a lot - no need to watch for
> SSL_WANT_READ and other issues, no need to switch watcher state based on
> return codes etc.

As you have to call SSL_write again with the same arguments again if you 
get a SSL_WANT_* error, you need two write queues, so you can provide 
the same arguments.

I was looking for rate limiting, and therefore I calculate how much I'm 
allowed to read/write.
To prevent reading/writing too small chunks, I allow reading/writing 
some bytes more - the amount is calculated relative to the allowed speed 
and already sent data for the sliding window-, and suspend the 
read/write io watchers in case I read/write to much, and use timers to 
wake up again.

As there are many ways to use non-blocking openssl in a wrong way, I'm 
glad it works.

Proper openssl shutdown was a real problem to me, as it did not work at 
all, turned out the problem was within openssl, and got patched in 0.9.8m.

I was looking at using BIOs myself, but I was not sure about rate 
limiting when using bios, and there is 'little' examples/documentation, 
besides some ascii image to illustrate how to design openssl nonblocking 
with bios.

> I wanted to make a C example for quite some time, but didn't get to it
> yet.

I saw libevent2 providing (rate limited) non-blocking openssl, but I'd 
say there is some (more) feature creep in libevent2, which makes it 
(even) less attractive to me.
Not even talking about performance - I got no numbers on libevent2, I 
prefer a simple api over features (I do not need in most cases).

But example code for (maybe even rate limited?) nonblocking ssl with 
bios on top of libev would be great.

More information about the libev mailing list