curl-library
Perl multicurl
Date: Thu, 7 May 2009 20:39:48 +0200 (CEST)
Hi,
I'm trying to do a multicurl request which behaves a bit odd.
When I 'perform' the multicurl handle, it seems all easy_handles are executed
with the URL of the last added handle. I tried to circumvent this problem by
POSTing instead, but somehow, the POST params don't get attached (But for this
I may be doing something wrong).
So, I've got two questions:
1/ what's messing with the passed url in the easy_handles?
2/ how can I attach POST params? I've tried two ways, just look at the
following code, without success. But I think both should work, sort of.
(3/ constants like CURLOPT_COPYPOSTFIELDS don't seem to work either)
I'm using the packaged perl from RH5, version 5.10.0 and the latest
WWW::Curl::Easy and Multi from CPAN, version 4.06.
Here's the code:
package CThreads;
use strict;
use warnings;
use URI::Escape;
use WWW::Curl::Easy;
use WWW::Curl::Multi;
#use Data::Dumper;
use constant SHAMAN_URL => 'http://127.0.0.1/cgi-app/';
sub new {
my $self = shift;
my $params = shift;
my $callback = $params->{ callback } || sub { return };
my $class = ref $self || $self;
# init the queue
my %q = (); # handle queue
my %response_of = (); # response queue
# init the multi object
my $cm = WWW::Curl::Multi->new;
my $active_handles = 0;
# init this object
$self = {
q => \%q,
response_of => \%response_of,
cm => $cm,
active_handles => $active_handles,
callback => $callback,
};
bless $self, $class;
return $self;
}
sub push {
my $self = shift;
my $uri = shift;
my $params = shift || { }; # a hash of params for the CGI script to be
called
my $id = $self->{ active_handles };
$id++;
# create the cURL handle
my $handle = WWW::Curl::Easy->new;
$handle->setopt(CURLOPT_URL , SHAMAN_URL . $uri);
$handle->setopt(CURLOPT_POSTFIELDS,
$self->_create_query_string($params));
#$handle->setopt(CURLOPT_POSTFIELDS, $params);
$handle->setopt(CURLOPT_POSTFIELDSIZE, -1); # please do a strlen for
me
$handle->setopt(CURLOPT_PRIVATE , $id); # add the reference number
$handle->setopt(CURLOPT_VERBOSE, 1);
# add a hook to catch the output
my $response;
open (my $fileb, ">", \$response);
$self->{ response_of }->{ $id } = \$response;
$handle->setopt(CURLOPT_WRITEDATA, $fileb);
# add the handle to the internal queue (we don't really use this,
except for Garbage Collection)
$self->{ q }->{ $id } = $handle;
$self->{ active_handles } = $id;
# add the handle to the multihandle
$self->{ cm }->add_handle($handle);
}
sub execute {
my $self = shift;
my $cm = $self->{ cm };
my $callback = $self->{ callback };
while ($self->{ active_handles }) {
if ($cm->perform != $self->{ active_handles }) {
while (my ($id, $return_value) = $cm->info_read) {
if ($id) {
$self->{ active_handles }--;
my $response = $self->{ response_of }->{ $id };
&{ $callback }( $response ); # do something with the
response
delete $self->{ q }->{ $id }; # Garbage Collect
}
}
}
}
}
sub _create_query_string {
my $self = shift;
my $params = shift;
my $query_string = q{};
while (my ($name, $value) = each %{ $params }) {
$query_string .= uri_escape($name) . '=' . uri_escape($value) .
'&';
}
return substr($query_string, 0, length($query_string) - 1); # return
it without trailing &
}
1;
And to test it I simply do:
#!/usr/local/bin/perl
use CThreads;
my $th = CThreads->new( { callback => sub{ use Data::Dumper; print Dumper
$_[0] } } );
$th->push('testmulticurladd.pl', { 'number' => 3 });
$th->push('testmulticurladd.pl', { 'number' => 5 });
$th->push('testmulticurladd.pl', { 'number' => 7 });
$th->push('testmulticurladd.pl', { 'number' => 9 });
$th->push('testmulticurladd.pl', { 'number' => 11 });
$th->execute;
And finally the testmulticurladd.pl cgi script is:
#!/usr/local/bin/perl
use CGI;
my $cgi = CGI->new;
print $cgi->header;
print $cgi->param('number') + 1;
I've googled around a lot, but it seems perl en cURL don't do a happy dance
together.
I'm a bit confused if WWW::Curl::Easy and WWW:Curl::Multi are still maintained,
and if they are recommended to be used or not. If you briefly google the
maillist archives, it's even mentioned that the curl package for perl is not
really as steadily maintained as their python equivalents. Is this still the
case?
Anyway, any help is appreciated,
thx,
Kenny
-- ================================================================== Kenny Billiau Web Developer Tel:+32 (0)9 331 36 95 fax:+32 (0)9 3313809 VIB Department of Plant Systems Biology, Ghent University Technologiepark 927, 9052 Gent, BELGIUM kenny.billiau@ugent.be http://bioinformatics.psb.ugent.be ================================================================== "Never get out of the boat" -- Chef, APNReceived on 2009-05-07