Async iterators

Marc Lehmann schmorp at
Wed Aug 8 00:50:07 CEST 2012

On Tue, Aug 07, 2012 at 10:06:05AM +0200, Clinton Gormley <clint at> wrote:
> >          # if we want more:
> >          $s->next (100, $processthem);
> Doesn't this end up with a lot of recursion?

No, because next should not invoke the callback, it should request more
results, and only when they are ready (long after next returns), it should
call the callback.

If you have the problem that you already have more results, you have to aks
yourself the question: if you have more results, why not pass the to the callback in the first
if it's unavoidable (e.g. because some server sends more results while the
callback processes them and you keep reading them) then you can use
AE::postpone to posepone the callback invocation.

That way, the stack depth has a maixmum (event loop => watcher => your
module => callback).

> > Variations would be to pass one or more results to the callback, or having
> > the callback return a boolean to cancel or continue the search (or a
> > number of further results to return).
> ... or providing a method which can be called to cancel further pulls.

Or that, if they come automatically without having to request them.

> Again, I'm not sure how to trigger the "pull new results if needed"
> without ending up with massive recursion.

It's not normally easy to trigger massive recursion, because normally you
pass all results to the callback, after which you haven't any. If you are
able of pulling in more results while the callback is processing them, then
you have to aks yourself if this is really sensible - why request more
results when the user has shown no interest (yet)? You could for example
abstain from reading more results unless requested.

The only case where the problem theoretically occurs is when you actively
read more results while the callback is processing the previous batch. Only
in that case would next be able to instantly recurse into the callback.

In most cases, this is not actually so desired, but when it is (for example,
because search results come in very slow and the server has to be polled
often to not pause), then you can AE::postpone the call to another "empty

However, you should first check whether this case should actually occur -
in most cases, not processing more results until the user has requested
them should be best.

                The choice of a       Deliantra, the free code+content MORPG
      -----==-     _GNU_    
      ----==-- _       generation
      ---==---(_)__  __ ____  __      Marc Lehmann
      --==---/ / _ \/ // /\ \/ /      schmorp at
      -=====/_/_//_/\_,_/ /_/\_\

More information about the anyevent mailing list