Starvation issue with Coro and AnyEvent

[Laurent]

Hi,

I am using Coro and AnyEvent and am facing a starvation problem.
The issue arises when 2 coros with the same priority are running. The
starved one is waiting on an AnyEvent event (sleep), while the greedy one
never waits, but calls cede periodically.

The issue is that the "starved" coro never wakes up when the "greedy" one
runs.

I created a test.pl program (copied below) to showcase this.
The starving coro displays '.' every 0.1s while the greedy coro displays
'-' or '+' at the same rate. '+' means the starving coro is in the ready
queue, '-' that it is not.
The expected result is to have interleaved '.' and '+' / '-' between the
"Start eating..." and "done" lines. When the issue happens, we just have
'-'s.

* When the starving coro uses Coro::AnyEvent::Sleep and the greedy coro
uses an active wait, the issue happens:

c:\test>perl test.pl 0
test: 0
Default priority
loop prio:     0
starving prio: 0
greedy prio:   0
.
Start eating...
----------
done


* When the greedy coro has a lower priority, there is no issue:

c:\test>perl test.pl 1
test: 1
loop prio:     0
starving prio: 0
greedy prio:   -1
.
Start eating...
-.-.-.-.-.-.-.-.-.-.
done


* When the loop coro has a higher priority, the starved coro is called once
(!):

c:\test>perl test.pl 2
test: 2
loop prio:     1
starving prio: 0
greedy prio:   0
.
Start eating...
-+.--------
done


* When the starved coro has a higher priority, the issue persists (the
starving coro fails to enter the ready queue):

c:\test>perl test.pl 3
test: 3
loop prio:     0
starving prio: 1
greedy prio:   0
.
Start eating...
----------
done


* When both loop and starved coro have a higher priority, there is no
issue. This is basically the same test than the 2nd one.

c:\test>perl test.pl 4
test: 4
loop prio:     1
starving prio: 1
greedy prio:   0
.
Start eating...
-.-.-.-.-.-.-.-.-.-.
done


* When the greedy coro uses sleep(0), there is no issue:

c:\test>perl test.pl 5
test: 5
Use sleep(0) to cede
loop prio:     0
starving prio: 0
greedy prio:   0
.
Start eating...
-+.+.+.+.+.+.+.+.+.
done


* Not is the test, but with 2 greedy coros, there is no issue either.


So there are ways to fix this issue at the application level, but I wander
if this is really the expected behavior from the coro perspective?
The coro scheduler is supposed to cede to a coro with same or higher
priority, that is longest in the ready queue. So I would have expected that
the greedy coro would cede to the evloop coro. EV would put the starving
coro back in the ready queue when the time event expired, and then the
scheduler would select the starving coro.


Also, among application-level fixes, what would be the recommended one?
- Using sleep(0) (or other event-loop level noop) as a replacement for cede
seems quite sub-optimal.
- Temporarily lowering the coro priority during the greedy work? There is a
risk for the greedy one to starve then. Alos, that could make things
difficult when debugging other starvation issues, as coro priorities would
change on the sly for some greedy operations.
- other solution?


For that matter, in the real life, the greedy coro performs a disk file
copy, on Windows (that is without AIO available). Using open() with perlio
':cede(0.01)' (as Coro::Storable does) does not help.

Thanks,


use common::sense;
use EV;
use AnyEvent;
use Coro;
use Coro::AnyEvent;

my $test = $ARGV[0] || 0;
print "test: $test\n";

AnyEvent::detect;
my $loop = async { EV::loop; };

my $starving = async {
  while () {
    print STDERR '.';
    Coro::AnyEvent::sleep(0.1);
  }
};

my $cv = AE::cv;
my $greedy = async {
  print "\nStart eating...\n";
  for (my $i=10; $i>0; $i--) {
    my $t = AE::time + 0.1;
    while (AE::time < $t) {
      # Active wait.
    }
    print STDERR ($starving->is_ready ? '+' : '-');
    if ($test == 5) {
      Coro::AnyEvent::sleep(0);
    } else {
      Coro::cede_notself;
    }
  }
  print "\ndone\n";
  $cv->send;
};

if ($test == 0) {
  print "Default priority\n";
} elsif ($test == 1) {
  # Low prio for starving coro.
  $greedy->prio(-1);
} elsif ($test == 2) {
  # High priority for loop.
  $loop->prio(1);
} elsif ($test == 3) {
  # High priority for starving.
  $starving->prio(1);
} elsif ($test == 4) {
  # High priotity for both loop and starving.
  $loop->prio(1);
  $starving->prio(1);
} elsif ($test == 5) {
  print "Use sleep(0) to cede\n";
}
print "loop prio:     ".$loop->prio()."\n";
print "starving prio: ".$starving->prio()."\n";
print "greedy prio:   ".$greedy->prio()."\n";

$cv->recv;
exit 0;
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.schmorp.de/pipermail/anyevent/attachments/20140123/eae06c91/attachment.html>