AnyEvent::Fork clarification

[Marc Lehmann]

On Mon, Jan 06, 2014 at 05:41:36PM -0500, Vikas N Kumar <vikas at vikaskumar.org> wrote:
> There is no <code>$cv->send </code> being called either.

In fact, calling the $cv as if it were a callback invokes send, which is
what happens internally in AnyEvent::Fork.

> The forking happens but I want the master/parent process to wait for all
> the forks to finish before it exits.

Right, waiting for the fh to close is the only thing you can do, at least,
without implementing anything extra.

In fact, waiting for the fh to close isn't even enough (another process could
have it open), but it's the only thing AnyEvent::Fork does for you.

> Here is what I am doing, and maybe you can suggest a better way.

>          exec @cmd;

In general, you should add "require POSIX; POSIX::_exit (1)" or so after
every exec when you don't handle errors from it. In this simple case,
nothing harmful should happen when exec returns though.

> 		push @watchers, AnyEvent->io(fh => $fh, poll => "w", cb => sub {

I have no clue how that would ever work for you - you wait for the handle to
become writable, which it should be immediately.

What you should do is to wait for it to become readable - if your code
doesn't write to the socket, it will become readable only when all processes
started by your AnyEvent::Fork process end, which closes the other side of
the handle, which makes it readable.

Now, the subprocess side of the handle might or might not be set to
close-on-exec (AnyEvent::Fork doesn't normally touch it), so in addition you
should mark the handle as not-close-on-exec ("use Proc::FastSpawn; fd_inherit
$fh, 1").

This will have the effect of any process you exec (and any process that
process execs) to inherit the fd, which will therefore stay open as long
as at leats one of the processes started is still running, which might be
better (or worse) then waiting for the exec'ed child.

Now, in general, here's why you should use AnyEvent::Fork instead of other
solutions:

1 when you want your code to be portable and it uses pthreads (for example,
  if IO::AIO is installed then AnyEvent might load it in the background,
  creating some threads as side effect).
2 when you want to fork faster (because it is faster to fork an
  AnyEvent::Fork process than to fork directly).
3 when you want to do actual work in the forked process without having to
  think about destructors, shared resources such as handles and so on.

I would say 3 doesn't apply to you, and 2 probably neither (but faster is
never wrong), and I don't know at all if 1 applies to you.

If 1 applies to you, then alternatives such as Parallel::ForkManager are all
asking for instabilities, as they all call fork directly, which has a high
chance of working - but only a high chance. In this case you would need to
find a way to do it with AnyEvent::Fork.

If it is important for you to actually wait for the child, you could use a
hybrid strategy: use AnyEvent::Fork to fork a new process, but do not exec
inside, but instead use e.g. "system":

   # forked child:
   use Proc::FastSpawn;
   fd_inherit $fh, 0; # set to cloexec
   my $status = system @cmd;
   syswrite $fh, $status;

   # parent:
   push @watchers, AnyEvent->io(fh => $fh, poll => "r", cb => sub {
      my $status = <$fh>;
      # got exit status here
   };

This is a bit convoluted, but forking twice (the second time for
system) is sitll faster than forking once from a big process, it is
guaranteed to work (there will be no threads or other resources such as
fd_cloexec-filehandles around for example), and you still find out when the
process you started finished, regardless of how many subprocesses it has
started.

Which might or might not be exactly what you want.

-- 
                The choice of a       Deliantra, the free code+content MORPG
      -----==-     _GNU_              http://www.deliantra.net
      ----==-- _       generation
      ---==---(_)__  __ ____  __      Marc Lehmann
      --==---/ / _ \/ // /\ \/ /      schmorp at schmorp.de
      -=====/_/_//_/\_,_/ /_/\_\