libeio EIO_STACKSIZE

[Konstantin Osipov]

* Marc Lehmann <schmorp at schmorp.de> [13/01/24 11:18]:

> On Wed, Jan 23, 2013 at 06:11:20PM +0400, Konstantin Osipov <kostja at tarantool.org> wrote:
> > * Marc Lehmann <schmorp at schmorp.de> [13/01/23 17:50]:
> > > Note also that the solution to having your own threadpool is simply...
> > > having your own threadpool.
> > 
> > Forking libeio is and (has always been with free software)
> > an obvious option.
> 
> Could you explain how forking libeio is an obvious option? I
> don't think you even understand the problem - your problem is
> not a shortcoming in libeio, or a missing option. Your problem
> is mixing incompatible request types.

Today, file I/O in Tarantool is done in a single dedicated
thread, which only job is to maintain an append-only file.
There is a performance win when writing to a single file from a
single thread with write() vs. writing to a single file from a
thread pool with pwrite() (apparently threads don't compete for
file position).

*So, at this point, Tarantool is not using libeio for async I/O at
all*.

But we still have some rare, non performance-critical
requests, which should not block the main libev event
loop, such as  getaddrinfo(). 

For these we need a reliable thread-pool of "gophers", i.e.
performers of random tasks, and libeio eio_custom fits here pretty
well.

When Tarantool will require reliable, efficient async I/O, 
we'll probably make it possible to have multiple libeio instances,
such as today it is possible to have multiple libev loops.
In any case, we'll cross this bridge when we come to it.

> Forking libeio will not solve that problem at all, it will just
> give you another libeio, which will then still be as unsuitable
> as the original libeio. Classical fail-fail.

There is a win for us since there is no other library out there
which can do async getaddrinfo() and integrate well into libev
event loop. There is a win for libeio/libev since we'll keep
reading the code and finding bugs in it.

> > a lot of time, and will save even more, should we decide to use
> > something else or write/fork.
> Libeio saves you time if you decide to use something else(?)

Yes, since it broadens one's perception: you can always
read the code, put it side by side, see the trade-offs, rather
than step on some obvious rakes trying to invent your own
solution.

I hope I was able to explain our case better. I think the second
part of email deserves a separate reply.

-- 
http://tarantool.org - an efficient, extensible in-memory data store