Async iterators

Marc Lehmann schmorp at
Mon Aug 6 19:17:11 CEST 2012

On Mon, Aug 06, 2012 at 06:35:10PM +0200, Clinton Gormley <clint at> wrote:
> This is what I'm hoping to get to. Just struggling to visualize what the
> code and usage actually looks like.

Well, here are a few examples of usages:

   my $s = new ElasticSearch::ScrolledSearch ...;

   my $processthem; $processthem = sub {
      my ($s, @results) = @_;

      if (@results) {
         for (@results) {

         # if we want more:
         $s->next (100, $processthem);
      } else {
         # search is done, make sure we don't leave refreences to ourselves
         # around
         undef $processthem;

   # start searching
   $s->next (100, $processthem);

That's similar to the blocking API.

Another way would be this:

   my $s = new ElasticSearch::ScrolledSearch
                  cb => sub {
                     my ($s, $result) = @_;


   # now you have the problem of storing $s somewhere for the duration of the
   # search.

Here the constructor just takes a callback that receives one result after

Variations would be to pass one or more results to the callback, or having
the callback return a boolean to cancel or continue the search (or a
number of further results to return).

Doing the latter makes it eaier to write simple callbacks, but often,
callbacks want to do some event-based stuff on their own, so "we need to know
whether to continue when you return" might be too limiting, and "just call is
any time later if you want more" (calling $s->next) is more flexible).

> I think I'm going to throw away what I've got and try to write it fresh,
> keeping the "don't call us, we'll call you" mantra in mind.

make sure existing users can still use the old way. one reason for this is
that breaking compatibility is bad (AnyEvent has taken great pains to keep
the api compatible over it's lifetime). another is that event-based apis are
definitely more complicated, cumbersome and write-intensive then simple
blocking apis might be. I cannot judge if this will be the case for your
current API, but maybe it is. At the very least, people might not be able to
get their mind around a callback api and this closure stuff.

so having a simple blockign api is useful on it's own.

you might even consider having an extra module such as
ElasticSearch::AnyEvent (or AnyEvent::ElasticSearch) and then base your
existing module on that one (they could also be packaged together of

                The choice of a       Deliantra, the free code+content MORPG
      -----==-     _GNU_    
      ----==-- _       generation
      ---==---(_)__  __ ____  __      Marc Lehmann
      --==---/ / _ \/ // /\ \/ /      schmorp at
      -=====/_/_//_/\_,_/ /_/\_\

More information about the anyevent mailing list