Can somebody throw some light on this. Is it true that php-curl binding does not scale for more than one request at a time ?? I am really curious

On 16-Mar-2011, at 6:12 AM, John Lange wrote:

> I have some code (written in PHP) that checks on the status of a
> "server" but I need this code to scale up to hundreds of parallel
> checks.
>
> For example, you call the script like this:
>
> http://mytest.com/checkserver.php?ip=10.11.12.13
>
> where the ip=X.X.X.X is different every time depending on which server
> you are checking.
>
> And the PHP for checkserver.php looks something like this:
>
> <?php
>
> $ch = curl_init();
> curl_setopt($ch, CURLOPT_URL, "http://".$ip."/status.html");
> curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
> $output = curl_exec($ch);
> curl_close($ch);
>
> ?>
>
> To my way of thinking, this should scale easily. Lets say 200 people
> hit the page at the same time all passing different "ip=" values, this
> should check all 200 different IPs in parallel and return results.
>
> It doesn't. I wrote this little bash script to test it (yes I'm using
> command line curl to test php-curl, please don't be confused by that):
>
> ---
> #!/bin/bash
>
> IPS="10.18.136.20
> 10.18.136.21
> 10.18.136.22
> ... ( a hundred more IPs)"
>
> for IP in $IPS ; do
> curl -s http://mytest.com/checkserver.php?ip=$IP &
> done
> ---
>
> All the curl commands launch into the background as you expect and I
> see all the apache child threads startup, but the results return only
> one by one. It's like each php-curl somehow blocks all the others.
>
> I can't for the life of me figure out how this can be possible. Each
> apache thread should run it's PHP in a separate thread and return in
> parallel.
>
> The only thing I can think of is that php is tracking all the requests
> as being part of the same session and imposing some limit on the
> outbound curl requests? But there is no global configuration for
> php-curl so how would you overcome this?
>
> Just thought someone might have had to do something similar in the
> past and run into this.
>
> By the way, it's not an OS limit or anything like that because if I
> bypass php if I go direct with command line curl like this:
>
> for IP in $IPS ; do
> curl -s http://".$IP"/status.html &
> done
>
> It works just exactly as you'd expect, all results return in parallel
> so it's got to be a problem with either PHP or apache.
>
> --
> John Lange
> www.johnlange.ca
> _______________________________________________
> http://cool.haxx.se/cgi-bin/mailman/listinfo/curl-and-php

_______________________________________________
http://cool.haxx.se/cgi-bin/mailman/listinfo/curl-and-php
Received on 2011-03-16