curl-and-php
Re: multi curl working very slow when threads > 10
Date: Mon, 24 Jan 2011 17:21:57 +0100 (CET)
On Mon, 24 Jan 2011, far Fraser wrote:
> Curl: 7.21.3
Using which resolver backend in libcurl?
> I'm trying simple multi curl script (GET list of URLs in 10-100 threads).
The libcurl multi interface doesn't use threads at all. It does multiple
transfers in a single thread.
> It does work on any windows, but when I try the same code on CentOS: if
> threads = 10, everything works almost fine, but when I set threads = 50 or
> 100 the process is almost stuck, it takes minutes for "multi_curl_exec"
That sounds like you're using the stock resolver, as that is synchronous all
the other transfers stall during each name resolve. 100 name resolves in
serial can take a long time. libcurl for windows defaults to build with a
threader resolver.
-- / daniel.haxx.se _______________________________________________ http://cool.haxx.se/cgi-bin/mailman/listinfo/curl-and-phpReceived on 2011-01-24