cURL / Mailing Lists / curl-library / Single Mail

curl-library

RE: Many CLOSE_WAIT when handling lots of URLs

From: Shao, Shuchao <sshao_at_websense.com>
Date: Mon, 17 Feb 2014 06:10:48 +0000

> > That means default 10 is make no sense? Can we just set the value as
> > the 4 times of the easy_handle number if this value is not set by user?
>
> Yes, I think that's sensible.

The attached patch is just to make sure the maxconnects limit works.

> MAXCONNECTS only limits how many connections that are kept in the cache
> _after_ they've been used. So if you set the limit to 50 and then add a 1000
> easy handles, you will still use 1000 connections but after all of them are
> done, libcurl should have no more than 50 connections in the connection
> cache.
>
> See also CURLMOPT_MAX_TOTAL_CONNECTIONS for a more strict limit.
 
Because the CURLMOPT_MAX_TOTAL_CONNECTIONS is set as unlimited as default, it is still easy to cumulate CLOSE_WAIT connections when the number of connections is too much.
Perhaps we have two options here,
1. Set a proper default value for CURLMOPT_MAX_TOTAL_CONNECTIONS, but not just unlimited
2. Use maxconnects to limit both done connections and in-use connections in cache, but not only limit the done connections as now. That's to say, in ConnectionDone(), we need to kill as many as idle connections in cache to make the number of connection is cache closer to maxconnects, but not just kill one idle connection as now.
Do you think we need a fix around here?

-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-library
Etiquette: http://curl.haxx.se/mail/etiquette.html

Received on 2014-02-17