Capturing the output of Multiple parallel http requests

[Marc Lehmann]

On Fri, Jan 28, 2011 at 12:30:13PM +0000, Ian Stuart <Ian.Stuart at ed.ac.uk> wrote:
> I can fire off multiple requests, and I can see the response from
> each of those queries.... but I can't get my head round how to
> return those responses back to my core code.

The question is, of course, where your core code is :)

>         'User::Agent'         => 'OA-RJ Broker v0.2',

(and this is a ^^ bug).

> How do I run a bunch of queries, and then capture each response?

If I understand you correctly, then you don't care whether you get the
responses when they arrive, but you want to process them in a non-event-based
fashion, basically:

phase 1: fire off requests
phase 2: wait for responses
phase 3: process responses

You can do something like that (with phase 2 and 3 combined) by using a
condvar for each request:

   sub transfer($) {
      ...
      my $cv = AnyEvent->condvar;

      http_request POST => ...,
         ...,
         $cv;

      $cv
   }

   for my $cv (map transfer $url, @my_urls) {
      my ($body, $hdr) = $cv->recv;
      ...
   }

Basically, transfer returns an objetc that you cna query for results - if
results are already there, then ->recv will just reutnr them, otherwise it
will wait for them.

The for loop then calls transfer for each of @my_urls, and then asks the
condvars it returns in order for results.

That way you get the results in the same order that you started the
requests, regardless of how they actually arrive form the network.

> 
> -- 
> 
> Ian Stuart.
> Developer: Open Access Repository Junction and OpenDepot.org
> Bibliographics and Multimedia Service Delivery team,
> EDINA,
> The University of Edinburgh.
> 
> http://edina.ac.uk/
> 
> This email was sent via the University of Edinburgh.
> 
> The University of Edinburgh is a charitable body, registered in
> Scotland, with registration number SC005336.
> 
> 
> _______________________________________________
> anyevent mailing list
> anyevent at lists.schmorp.de
> http://lists.schmorp.de/mailman/listinfo/anyevent

-- 
                The choice of a       Deliantra, the free code+content MORPG
      -----==-     _GNU_              http://www.deliantra.net
      ----==-- _       generation
      ---==---(_)__  __ ____  __      Marc Lehmann
      --==---/ / _ \/ // /\ \/ /      schmorp at schmorp.de
      -=====/_/_//_/\_,_/ /_/\_\