libeio EIO_STACKSIZE

[Marc Lehmann]

On Wed, Jan 23, 2013 at 06:11:20PM +0400, Konstantin Osipov <kostja at tarantool.org> wrote:
> * Marc Lehmann <schmorp at schmorp.de> [13/01/23 17:50]:
> > Note also that the solution to having your own threadpool is simply...
> > having your own threadpool.
> 
> Forking libeio is and (has always been with free software)
> an obvious option.

Could you explain how forking libeio is an obvious option? I don't think you
even understand the problem - your problem is not a shortcoming in libeio, or
a missing option. Your problem is mixing incompatible request types.

Forking libeio will not solve that problem at all, it will just give you
another libeio, which will then still be as unsuitable as the original
libeio. Classical fail-fail.

> a lot of time, and will save even more, should we decide to use
> something else or write/fork.

Libeio saves you time if you decide to use something else(?)

> > stack size, and EIO_STACKSIZE is simply an additional rarely useful
> > feature. Using it for gethostbyname or similar functions is already a
> > design bug, and only useful for hacks.
> 
> Do you care to elaborate why?

Obviously yes, which is why I elaborated on this multiple times, but it
seems I can't make you understand. At this point, you should probably explain
carefully what you don't understand about the explanations.

> Yet I have to admit I'm still clueless at what the real technical issue
> is.

Me too - you are the one who seems to insist it is a technical issue, despite
me telling you this is not so.

> Is there any other reason why using libeio with host name resolution is a
> hack, apart from the necessarily large stack size?

The large stack size requireemnt is not a hack, it's inconvenient, possibly
ugly.

As for the reasons, I have stated them multiple times, including in the
thread you linked to.

> Personally, I'm not bothered much with need for a large stack
> size, since on most modern hardware even 8M of stack is only
> 8M of virtual address space, which still leaves me with hundreds
> of millions of threads (assuming 64-bit pointers).

While this is completely unrelated, this is wrong on so many levels:

1. 64 bit pointers don't give you millions of stacks, they are a requirement,
   but not sufficient. For example, on very common amd64 hardware your address
   space is enough for ~16 million stacks of 8MB size, which is a far cry from
   hundreds of millions. And that's already quite generous for 64 bit
   architectures.
2. If it truly were virtual, then you could use a much smaller stack space.
   The fact that you get crashes indicates that the stack is "physically"
   too small, nothing cares about virtual size.
3. This is only true when overcommitting 100%, which is quite uncommon.
4. It disregards the memory used for page tables - for your hundreds of
   millions of threads you would require gigabytes of ram without
   counting thew actual stack itself.

If you base your decisions on the kind of myths you state above you will
eventually be very surprised.

> > Anything you seem to want out of it is likely not what eio is designed
> > for, and will interfere with libeios normal operation in negative
> > ways.
> 
> There is always a possibility that the author meant one thing, and
> the reader is seeing something the author did not intend.

Sure, but quite obviously libeio using gethostbyname is not one of these
things. The author of that library is just trying to improve your life by
keeping you from making a big mistake that will likely cause trouble.

> So the real question is: what this library is good for today,
> what are future directions, and how having (or not having) the
> subject feature helps or harms these and existing purposes.

Good that all these questions have already been answered then.

> > My copy of eio.pod clearly mentions the name of the constant and it's
> > description, just in the section you quoted.
> 
> I'm not looking at eio.pod, I'm looking at this URL:
> 
> http://pod.tst.eu/http://cvs.schmorp.de/libeio/eio.pod

Ah, something is clearly going wrong with the pod => html conversion, thanks
for pointing this out.

-- 
                The choice of a       Deliantra, the free code+content MORPG
      -----==-     _GNU_              http://www.deliantra.net
      ----==-- _       generation
      ---==---(_)__  __ ____  __      Marc Lehmann
      --==---/ / _ \/ // /\ \/ /      schmorp at schmorp.de
      -=====/_/_//_/\_,_/ /_/\_\