NXWEB - ultra-fast web server based on libev

Yaroslav yarosla at gmail.com
Sun Dec 4 23:08:11 CET 2011 

I have published new benchmarks:
https://bitbucket.org/yarosla/nxweb/wiki/Benchmarks

On Sun, Dec 4, 2011 at 6:00 PM, Yaroslav <yarosla at gmail.com> wrote:

> My ulimit I think is high enough. It was 65536, now I checked with 131072.
> Same results. And with all three servers: nxweb, g-wan, nginx.
>
> But after applying magic sysctl script error rate actually reduced. It
> looks frightening though.
>
>
> On Sun, Dec 4, 2011 at 5:26 AM, Zabrane Mickael <zabrane3 at gmail.com>wrote:
>
>> Hi Yaroslav,
>>
>> I noticed te 104 error only when the server can't handle the large amount
>> of connections correctly.
>>
>> Did you check your *ulimit* settings?
>> You can refer to the excellent G-WAN doc explaining how to tweek your
>> seetings at:
>> http://gwan.ch/source/ab.c.txt
>>
>> Let me give *httpress* a try, I'll be back with more info.
>>
>> Keep up the good work man.
>>
>> Regards,
>> Zabrane
>>
>> On Dec 4, 2011, at 1:25 AM, Yaroslav wrote:
>>
>> Hi, everybody!
>>
>> I tried to use weighttp for benchmarking but encountered some problems
>> when using large number of concurrent connections in multi-threaded mode.
>> For example:
>>
>>    weighttp -c 10000 -n 100000 -t 4 -k http://...
>>
>> This almost always produces quite a number of '104 Connection reset by
>> peer' errors, many connections do not close, which means the test runs
>> long, and measurements get poor. This is not specific to nxweb, same
>> situation with all tested servers. I even tried it on different hardware
>> with CentOS. Same situation. The interesting part is that when used in
>> single thread mode (-t 1) there are no such errors even with large number
>> of connections.
>>
>> I've spent whole day digging the source of weighttp, even made some
>> patches, but could not figure out the source of the problem.
>>
>> Finally I've decided to write my own benchmarking tool, and successfully
>> did it. With the same result though... As I also used libev.
>>
>> Anyway I made some improvements over weighttp, namely:
>> - load is evenly distributed among threads, in weighttp one thread could
>> finish much earlier than others
>> - I forcibly cut timed out connections, not allowing them to spoil the
>> test
>>
>> Otherwise it is the same efficient as weighttp. Need to mention that I
>> have not implemented (yet) chunked encoding, custom headers and IPv6, which
>> weighttp implements.
>>
>> Here is the project page if anyone interested:
>> https://bitbucket.org/yarosla/httpress/
>>
>> I plan to redo all benchmarks for NXWEB in a few days. With the new tool
>> results are much better now.
>>
>> I wonder if anybody experienced same issues with hanging/reset
>> connections with libev under high concurrency?
>>
>> Regards,
>>
>> Yaroslav
>>
>> On Fri, Dec 2, 2011 at 3:14 PM, Yaroslav <yarosla at gmail.com> wrote:
>>
>>> Hi Mickael,
>>>
>>> I mentioned in limitations that it has only been tested on Linux. I
>>> can't test it on OSX unfortunately, and frankly speaking portability was
>>> not my primary goal. At least not at this stage. Hosting providers mostly
>>> offer Linux, never heard about OSX.
>>>
>>> But still I think it should be possible to port it to OSX, it's not
>>> Windows anyway.
>>>
>>> What causing compile errors is the absense of argp and obstack in OSX's
>>> library. argp is not so critical for NXWEB - this is just command line
>>> parsing. While obstack is used extensively by the core.
>>>
>>> Thanks for pointing to benchmarking tool. I will take a look at it. I
>>> wasn't satisfied by ab, but couldn't find anything better. httperf happened
>>> to be too complicated for my brain.
>>>
>>> Yaroslav
>>>
>>>
>>> On Fri, Dec 2, 2011 at 2:56 PM, Zabrane Mickael <zabrane3 at gmail.com>wrote:
>>>
>>>> Hi Yaroslav,
>>>>
>>>> My first try to compile NxWeb (OSX 10.6.8 + libev4):
>>>>
>>>> *nxweb* $  make
>>>> mkdir -p bin/Release
>>>> mkdir -p obj/Release/nxweb
>>>> gcc -c -o obj/Release/nxweb/main.o nxweb/main.c -pthread
>>>> -Wno-strict-aliasing -O2 -s
>>>> nxweb/main.c:29:18: error: argp.h: No such file or directory
>>>> In file included from nxweb/nxweb_internal.h:30,
>>>>                  from nxweb/main.c:35:
>>>> nxweb/nxweb.h:30:21: error: obstack.h: No such file or directory
>>>> In file included from nxweb/nxweb_internal.h:30,
>>>>                  from nxweb/main.c:35:
>>>> nxweb/nxweb.h:121: error: field ‘data’ has incomplete type
>>>> nxweb/nxweb.h:122: error: field ‘user_data’ has incomplete type
>>>> nxweb/main.c:123: error: array type has incomplete element type
>>>> nxweb/main.c:140: error: expected ‘=’, ‘,’, ‘;’, ‘asm’ or
>>>> ‘__attribute__’ before ‘parse_opt’
>>>> nxweb/main.c:165: error: variable ‘argp’ has initializer but incomplete
>>>> type
>>>> nxweb/main.c:165: warning: excess elements in struct initializer
>>>> nxweb/main.c:165: warning: (near initialization for ‘argp’)
>>>> nxweb/main.c:165: error: ‘parse_opt’ undeclared here (not in a function)
>>>> nxweb/main.c:165: warning: excess elements in struct initializer
>>>> nxweb/main.c:165: warning: (near initialization for ‘argp’)
>>>> nxweb/main.c:165: warning: excess elements in struct initializer
>>>> nxweb/main.c:165: warning: (near initialization for ‘argp’)
>>>> nxweb/main.c:165: warning: excess elements in struct initializer
>>>> nxweb/main.c:165: warning: (near initialization for ‘argp’)
>>>> make: *** [obj/Release/nxweb/main.o] Error 1
>>>>
>>>>
>>>> I think you should update your *benchmark* to use *weighttp*:
>>>> http://redmine.lighttpd.net/projects/weighttp/wiki
>>>> As pointed by G-WAN's author, *ab* performs badly on multi-cores.
>>>>
>>>> Regards,
>>>> Zabrane
>>>>
>>>> On Dec 2, 2011, at 9:58 AM, Yaroslav wrote:
>>>>
>>>> What a shame. Sorry about that. It is my first public repo at
>>>> Bitbucket, I turned 'private' flag off, but did not check that I have to
>>>> make wiki and issue tracker public as well.
>>>>
>>>> It should be all OK now.
>>>>
>>>> Sorry for the inconvenience.
>>>>
>>>> On Fri, Dec 2, 2011 at 12:13 PM, Chris Brody <chris.brody at gmail.com>wrote:
>>>>
>>>>> Yeah I created an account with Bitbucket and still could not get
>>>>> access to the Wiki.
>>>>>
>>>>> Can you please make this project public if you want to announce it?
>>>>>
>>>>>
>>>>> On Fri, Dec 2, 2011 at 3:04 AM, Rodrigo Campos <rodrigo at sdfg.com.ar>wrote:
>>>>>
>>>>>> On Fri, Dec 02, 2011 at 03:55:27AM +0400, Yaroslav wrote:
>>>>>> > Hi,
>>>>>> >
>>>>>> > I'd like to thank the authors of libev for very good product. Based
>>>>>> on it I
>>>>>> > have developed NXWEB web server, which I'd like to present to
>>>>>> community. Here
>>>>>> > are the highlights:
>>>>>> >
>>>>>> > NXWEB project page: https://bitbucket.org/yarosla/nxweb/
>>>>>>
>>>>>> Sounds interesting, but the project page asks for a login. It seems I
>>>>>> can not
>>>>>> see anything without an account :(
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> Thanks,
>>>>>> Rodrigo
>>>>>>
>>>>>> _______________________________________________
>>>>>> libev mailing list
>>>>>> libev at lists.schmorp.de
>>>>>> http://lists.schmorp.de/cgi-bin/mailman/listinfo/libev
>>>>>
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> libev mailing list
>>>>> libev at lists.schmorp.de
>>>>> http://lists.schmorp.de/cgi-bin/mailman/listinfo/libev
>>>>>
>>>>
>>>> _______________________________________________
>>>> libev mailing list
>>>> libev at lists.schmorp.de
>>>> http://lists.schmorp.de/cgi-bin/mailman/listinfo/libev
>>>>
>>>>
>>>>
>>>>
>>>
>> _______________________________________________
>> libev mailing list
>> libev at lists.schmorp.de
>> http://lists.schmorp.de/cgi-bin/mailman/listinfo/libev
>>
>>
>>
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.schmorp.de/pipermail/libev/attachments/20111205/62ff48c4/attachment-0001.html>

More information about the libev mailing list