[ANN] x0 HTTP server and framework (initial release)

[common at gmx.ch]

Marc Lehmann wrote:
> On Thu, Apr 01, 2010 at 10:53:39AM +0200, Graham Leggett <minfrin at sharp.fm> wrote:
>> You might be tempted to keep two socket watchers going for listening to 
>> socket read events, and socket write events, but to support SSL  
> 
> This is the mess how it is commonly implemented, and it is indeed horrors.

I've had the non-blocking ssl challenge too, and I use two socket 
watchers, I know it can be done using a single watcher, but ... I'd call 
it a tradeoff, using two io watchers simplified the code and therefore I 
decided to take the performance impact into account.

In fact, given the possibility, I use two io watchers for every type of 
connection, even tcp and udp, and a multitude of timers for each 
connection type to provide timeouts for all kinds of things I want to be 
able to deal with (idle, sustain, listen, throtte_(in,out), connecting, 
dnsresolve, close, handshake, reconnect).

This makes a total of
  * 2 io watchers
  * 10 timers
-> 12 watchers per connection

where 1 io watcher (in) and 2 timers (idle&sustain) are active most of 
the time.

I'm totally aware could be done with 2 watchers, one for io, and one 
timeout and some internal list to have only the 'next' timeout active in 
libev, but I don't feel bad about it.

In fact, libev performs that great, I did not even notice a when it had 
about 6000 connections.

So, if you don't need the extra tidbit of performance, you can keep 
things simple.

> A much easier way is to decouple openssl from the socket altogether,
> i.e. don't let it do socket I/O, instead let it do I/O from/two two
> BIO buffers, which simplifies things a lot - no need to watch for
> SSL_WANT_READ and other issues, no need to switch watcher state based on
> return codes etc.


As you have to call SSL_write again with the same arguments again if you 
get a SSL_WANT_* error, you need two write queues, so you can provide 
the same arguments.

I was looking for rate limiting, and therefore I calculate how much I'm 
allowed to read/write.
To prevent reading/writing too small chunks, I allow reading/writing 
some bytes more - the amount is calculated relative to the allowed speed 
and already sent data for the sliding window-, and suspend the 
read/write io watchers in case I read/write to much, and use timers to 
wake up again.

As there are many ways to use non-blocking openssl in a wrong way, I'm 
glad it works.

Proper openssl shutdown was a real problem to me, as it did not work at 
all, turned out the problem was within openssl, and got patched in 0.9.8m.

I was looking at using BIOs myself, but I was not sure about rate 
limiting when using bios, and there is 'little' examples/documentation, 
besides some ascii image to illustrate how to design openssl nonblocking 
with bios.

> I wanted to make a C example for quite some time, but didn't get to it
> yet.

I saw libevent2 providing (rate limited) non-blocking openssl, but I'd 
say there is some (more) feature creep in libevent2, which makes it 
(even) less attractive to me.
Not even talking about performance - I got no numbers on libevent2, I 
prefer a simple api over features (I do not need in most cases).

But example code for (maybe even rate limited?) nonblocking ssl with 
bios on top of libev would be great.