curl-library
RE: Many CLOSE_WAIT when handling lots of URLs
Date: Thu, 20 Feb 2014 02:48:39 +0000
> You make it sound as if our job is primarily to limit the number of
> CLOSE_WAIT connections. It really isn't.
>
> The CLOSE_WAIT connections are mostly (AFAIK) unfortunate side-effects of
> us keeping connections alive after we're done using them, and while kept
> like that they get closed from the other end.
Many CLOSE_WAIT connections will impact the libcurl performance significantly. Every system has a limit for maximum number of open socket, the more CLOSE_WAIT connection will result in less ESTABLISHED connections.
> I'm thinking we can change the if() in ConnectionDone() to a while() instead
> and kill off connections until below the threshold. We should also add a test
> case for that.
Just changing if() to while() seems does not have a lot help.
In my setup,
Using libcurl-7.24.0, I can have about 20K ESTABLISHED connections, no CLOSE_WAIT connections. Using 7.35.0 (after changing if() to while()), the number will be 9K for ESTABLISHED and 11K for CLOSE_WAIT.
I am wondering if we need to close the connections for a easy_handle when we removing it from a multi_handle?
-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-library
Etiquette: http://curl.haxx.se/mail/etiquette.html
Received on 2014-02-20