cURL / Mailing Lists / curl-library / Single Mail

curl-library

Re: Curl downloading timeouts/speed issues with many concurrent downloads?

From: Daniel Stenberg <daniel-curl_at_haxx.se>
Date: Wed, 6 Jul 2005 00:22:13 +0200 (CEST)

On Tue, 5 Jul 2005, Michael Mastroianni wrote:

> I'm using 12.3.1

Wow, you must be living in a distant future! 7.14.0 is the most recent
released version...

I figure you mean 7.12.3-1 or something?

> I have an application which concurrently downloads many urls using the easy
> interface on lots of threads (for various reasons, the multi interface isn't
> something I can easily accommodate.). Anyway, on one of our linux boxes, my
> application times out 50% of the time, and gets DNS failures 21% of the time
> (this is pretty rare on our windows boxes). But if I wget these urls on the
> command line on the machine, they download just fine.

You have any timeouts set in libcurl? If so, for how long?

What if you strace or similar (possibly even ethereal), would does that show
that libcurl does or doesn't do in the failing cases?

Are the failures evenly spread out over time? Do you get the problems even if
only one thread is running? If not, how many threads are required?

If you must use many threads to see the problem, do the problems still occur
if you slow down the rate of their operations?

> I'm trying to hunt down what flavor of Red Hat the thing is running.

I don't think that's very important. (but cat /etc/issue usually tells)

> Does this ring a bell with anyone? Has anyone else reported such behavior?

Not really and no. :-(

-- 
  Commercial curl and libcurl Technical Support: http://haxx.se/curl.html
Received on 2005-07-06