cURL / Mailing Lists / curl-library / Single Mail

curl-library

RE: Many CLOSE_WAIT when handling lots of URLs

From: Daniel Stenberg <daniel_at_haxx.se>
Date: Thu, 13 Feb 2014 08:46:10 +0100 (CET)

On Thu, 13 Feb 2014, Shao, Shuchao wrote:

>> So there's definately a fix needed here. The documentation could be improved
>> too as it seems stupid to say that the default is 10 but as soon as you add
>> one handle, it'll be 4...

> That means default 10 is make no sense? Can we just set the value as the 4
> times of the easy_handle number if this value is not set by user?

Yes, I think that's sensible.

> And what is the meaning if set the maxconnects as 0, means don't cache
> connection or cache all?

I'd say cache all. I rather not provide a way to completely shut off the cache
using this option. You can still easily achieve that effect by setting easy
options to stop re-use etc.

> In my test setup, adding 50 eash_handles to a multi-handle. If set the
> maxconnects <= 50, the cache connections number will always be about 45-50.

MAXCONNECTS only limits how many connections that are kept in the cache
_after_ they've been used. So if you set the limit to 50 and then add a 1000
easy handles, you will still use 1000 connections but after all of them are
done, libcurl should have no more than 50 connections in the connection cache.

See also CURLMOPT_MAX_TOTAL_CONNECTIONS for a more strict limit.

> I think the reproduce condition is keeping the connections number in cache
> as large enough, for example, >10000. Use the example I send on Jan 6
> (http://curl.haxx.se/mail/lib-2014-01/0033.html ) can reproduce it every
> time.

If there's no limit to the connection cache, that's of course a very good
explanation to why there's no upper bound to how many connections libcurl will
store there!

Let's start with making sure that the limit works as intended.

-- 
  / daniel.haxx.se
-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-library
Etiquette:  http://curl.haxx.se/mail/etiquette.html
Received on 2014-02-13