cURL / Mailing Lists / curl-library / Single Mail

curl-library

Perl multicurl

From: Kenny Billiau <kenny.billiau_at_ugent.be>
Date: Thu, 7 May 2009 20:39:48 +0200 (CEST)

Hi,

I'm trying to do a multicurl request which behaves a bit odd.
When I 'perform' the multicurl handle, it seems all easy_handles are executed
with the URL of the last added handle. I tried to circumvent this problem by
POSTing instead, but somehow, the POST params don't get attached (But for this
I may be doing something wrong).

So, I've got two questions:
1/ what's messing with the passed url in the easy_handles?
2/ how can I attach POST params? I've tried two ways, just look at the
following code, without success. But I think both should work, sort of.
(3/ constants like CURLOPT_COPYPOSTFIELDS don't seem to work either)

I'm using the packaged perl from RH5, version 5.10.0 and the latest
WWW::Curl::Easy and Multi from CPAN, version 4.06.

Here's the code:

     package CThreads;

      use strict;
      use warnings;
      use URI::Escape;
      use WWW::Curl::Easy;
      use WWW::Curl::Multi;

     #use Data::Dumper;

     use constant SHAMAN_URL => 'http://127.0.0.1/cgi-app/';

      sub new {
          my $self = shift;
          my $params = shift;
          my $callback = $params->{ callback } || sub { return };

         my $class = ref $self || $self;

          # init the queue
          my %q = (); # handle queue
          my %response_of = (); # response queue

          # init the multi object
          my $cm = WWW::Curl::Multi->new;
          my $active_handles = 0;

          # init this object
          $self = {
              q => \%q,
              response_of => \%response_of,
              cm => $cm,
              active_handles => $active_handles,
              callback => $callback,
          };

          bless $self, $class;
          return $self;
      }

      sub push {
          my $self = shift;
          my $uri = shift;
          my $params = shift || { }; # a hash of params for the CGI script to be
          called
          my $id = $self->{ active_handles };
          $id++;

          # create the cURL handle
          my $handle = WWW::Curl::Easy->new;
          $handle->setopt(CURLOPT_URL , SHAMAN_URL . $uri);
          $handle->setopt(CURLOPT_POSTFIELDS,
          $self->_create_query_string($params));
          #$handle->setopt(CURLOPT_POSTFIELDS, $params);
          $handle->setopt(CURLOPT_POSTFIELDSIZE, -1); # please do a strlen for
          me
          $handle->setopt(CURLOPT_PRIVATE , $id); # add the reference number
          $handle->setopt(CURLOPT_VERBOSE, 1);

          # add a hook to catch the output
          my $response;
          open (my $fileb, ">", \$response);
          $self->{ response_of }->{ $id } = \$response;
          $handle->setopt(CURLOPT_WRITEDATA, $fileb);

          # add the handle to the internal queue (we don't really use this,
          except for Garbage Collection)
          $self->{ q }->{ $id } = $handle;
          $self->{ active_handles } = $id;

          # add the handle to the multihandle
          $self->{ cm }->add_handle($handle);
      }

      sub execute {
          my $self = shift;
          my $cm = $self->{ cm };
          my $callback = $self->{ callback };

          while ($self->{ active_handles }) {
              if ($cm->perform != $self->{ active_handles }) {
                  while (my ($id, $return_value) = $cm->info_read) {
                      if ($id) {
                          $self->{ active_handles }--;
                          my $response = $self->{ response_of }->{ $id };

                         &{ $callback }( $response ); # do something with the
response

                          delete $self->{ q }->{ $id }; # Garbage Collect
                      }
                  }
              }
          }
      }

      sub _create_query_string {
          my $self = shift;
          my $params = shift;

          my $query_string = q{};
          while (my ($name, $value) = each %{ $params }) {
              $query_string .= uri_escape($name) . '=' . uri_escape($value) .
          '&';
          }

          return substr($query_string, 0, length($query_string) - 1); # return
      it without trailing &
      }

     1;

And to test it I simply do:

     #!/usr/local/bin/perl

     use CThreads;

     my $th = CThreads->new( { callback => sub{ use Data::Dumper; print Dumper
$_[0] } } );

      $th->push('testmulticurladd.pl', { 'number' => 3 });
      $th->push('testmulticurladd.pl', { 'number' => 5 });
      $th->push('testmulticurladd.pl', { 'number' => 7 });
      $th->push('testmulticurladd.pl', { 'number' => 9 });
      $th->push('testmulticurladd.pl', { 'number' => 11 });

         $th->execute;

And finally the testmulticurladd.pl cgi script is:

     #!/usr/local/bin/perl

     use CGI;

     my $cgi = CGI->new;

      print $cgi->header;
      print $cgi->param('number') + 1;

I've googled around a lot, but it seems perl en cURL don't do a happy dance
together.
I'm a bit confused if WWW::Curl::Easy and WWW:Curl::Multi are still maintained,
and if they are recommended to be used or not. If you briefly google the
maillist archives, it's even mentioned that the curl package for perl is not
really as steadily maintained as their python equivalents. Is this still the
case?

Anyway, any help is appreciated,
thx,
Kenny

-- 
==================================================================
Kenny Billiau
Web Developer
Tel:+32 (0)9 331 36 95                        fax:+32 (0)9 3313809
VIB Department of Plant Systems Biology, Ghent University
Technologiepark 927, 9052 Gent, BELGIUM
kenny.billiau@ugent.be          http://bioinformatics.psb.ugent.be
==================================================================
                         "Never get out of the boat" -- Chef, APN
Received on 2009-05-07