Thanks. You were of course correct. PHP was locking the session
preventing more than one result from executing at a time.

By modifying the curl at the command line so that it used a new cookie
for each request the problem was solved.

Thanks again.

John

On Wed, Mar 16, 2011 at 5:59 AM, Waruna Geekiyanage <warunans_at_gmail.com> wrote:
> Sorry, I got it incorrectly since it was obvious to me that curl_exec works
> fine in parallel in separate requests.
> It seems that the testing method you use got the problem. If you start a
> session (or auto starts) all the requests are in the same session.
> Session data is locked to prevent concurrent writes only one script may
> operate on a session at any time
> So until the session_write_close is called , no output will be given.
> check the script without starting the session or call session_write_close()
> before the curl_exec call .
>
> Waruna
>
> On 3/16/2011 1:28 PM, Kuberan Marimuthu wrote:
>
> Hi Waruna,
> His scenario is different. When more than one request is made to a PHP
> script that makes a curl_exec, will the HTTP request happen concurrently ?
> regards,
> Kuberan Marimuthu
> On 16-Mar-2011, at 1:20 PM, Waruna Geekiyanage wrote:
>
> Initialize curl. (curl_init, or curl_multi_init)
> Do thing(s). (in multiple curl_exec calls if needed, or curl_multi_exec)
> Close curl. (curl_close or curl_multi_close)
>
> http://www.php.net/manual/en/function.curl-multi-init.php
>
>
> On 3/16/2011 1:05 PM, Kuberan Marimuthu wrote:
>
> Can somebody throw some light on this. Is it true that php-curl binding does
> not scale for more than one request at a time ?? I am really curious
>
> On 16-Mar-2011, at 6:12 AM, John Lange wrote:
>
> I have some code (written in PHP) that checks on the status of a
> "server" but I need this code to scale up to hundreds of parallel
> checks.
>
> For example, you call the script like this:
>
> http://mytest.com/checkserver.php?ip=10.11.12.13
>
> where the ip=X.X.X.X is different every time depending on which server
> you are checking.
>
> And the PHP for checkserver.php looks something like this:
>
> <?php
>
> $ch = curl_init();
> curl_setopt($ch, CURLOPT_URL, "http://".$ip."/status.html");
> curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
> $output = curl_exec($ch);
> curl_close($ch);
>
> ?>
>
> To my way of thinking, this should scale easily. Lets say 200 people
> hit the page at the same time all passing different "ip=" values, this
> should check all 200 different IPs in parallel and return results.
>
> It doesn't. I wrote this little bash script to test it (yes I'm using
> command line curl to test php-curl, please don't be confused by that):
>
> ---
> #!/bin/bash
>
> IPS="10.18.136.20
> 10.18.136.21
> 10.18.136.22
> ... ( a hundred more IPs)"
>
> for IP in $IPS ; do
> curl -s http://mytest.com/checkserver.php?ip=$IP &
> done
> ---
>
> All the curl commands launch into the background as you expect and I
> see all the apache child threads startup, but the results return only
> one by one. It's like each php-curl somehow blocks all the others.
>
> I can't for the life of me figure out how this can be possible. Each
> apache thread should run it's PHP in a separate thread and return in
> parallel.
>
> The only thing I can think of is that php is tracking all the requests
> as being part of the same session and imposing some limit on the
> outbound curl requests? But there is no global configuration for
> php-curl so how would you overcome this?
>
> Just thought someone might have had to do something similar in the
> past and run into this.
>
> By the way, it's not an OS limit or anything like that because if I
> bypass php if I go direct with command line curl like this:
>
> for IP in $IPS ; do
> curl -s http://".$IP"/status.html &
> done
>
> It works just exactly as you'd expect, all results return in parallel
> so it's got to be a problem with either PHP or apache.
>
> --
> John Lange
> www.johnlange.ca
> _______________________________________________
> http://cool.haxx.se/cgi-bin/mailman/listinfo/curl-and-php
>
> _______________________________________________
> http://cool.haxx.se/cgi-bin/mailman/listinfo/curl-and-php
>
>
> --
>
> Thank You,
> Waruna Geekiyanage
>
> M: +94 71 4168164
> E: waruna.ns_at_gmail.com
> MSN: warunans_at_hotmail.com
> GTalk: waruna.ns
> ICQ: 225184383
> Yahoo: warunans
> Skype: warunans
>
> _______________________________________________
> http://cool.haxx.se/cgi-bin/mailman/listinfo/curl-and-php
>
>
>
> --
>
> Thank You,
> Waruna Geekiyanage
>
> M: +94 71 4168164
> E: waruna.ns_at_gmail.com
> MSN: warunans_at_hotmail.com
> GTalk: waruna.ns
> ICQ: 225184383
> Yahoo: warunans
> Skype: warunans
>
> _______________________________________________
> http://cool.haxx.se/cgi-bin/mailman/listinfo/curl-and-php
>
>

-- 
John Lange
www.johnlange.ca
_______________________________________________
http://cool.haxx.se/cgi-bin/mailman/listinfo/curl-and-php