sorry I didn't reply, I lost track of this thread somehow :-/

> What about running the thing every time a transfer completes? Or perhaps
> even we could do it at completion time and when a handle is removed from the
> multi handle.

that would not be of much help, because (at least in our case) the servers we are taking to have keepalive enabled, so the socket get's into close_wait after serveral seconds, when the process is already doing something else. And If there is a new request using curl_perform all the half-open connections in the connection cache will be closed anyway (since patch http://curl.haxx.se/mail/lib-2009-03/0009.html). So the Problem exists when a lot of backend connections are established, and later (few seconds) they are being closed by the peer. At that point in time I should be able to detect this and to close the connection from my side.

Generally in an application where I had to handle a lot of sockets, the (IMHO) right approach would be to put all this sockets (fdset) in a select, and depending on which one something is happending (close, accept, data to read) you can handle it appropreatly. In the meantime the application can suspend in the select, or do something else. So its possible to react "~immediatly" to actions on sockets, and otherwise the application does only run if necessary. Now with libCurl as it is now I don't see this kind of possibility (without extending the API, I'm afraid). Even if It would be possible to get an fdset of all the sockets in the connection cache, maybe by curl_easy_getinfo(), I'd need a function like curl_handle_socket() to handle the socket action (close read etc), just closing the socket, that is und curl's control wouldn't be nice I think.

Another approch I see, to cleanup the connection cache with each call to curl_multi_perform. Since the connection cache is attached to the multihandle, and curl_multi_perform does not block, it could be used as a cleanup function that has to be called from time to time (not so nice). An improvement could be only to cleanup the cache if no running easy_handles are attached to the multi_handle. In the easy_interface something similiar could be done if URL is set to NULL, curl_easy_perform() only cleans up the cache (like it does now if a URL is given).

As I see the extension of the API is not an option for you, so I would suggest to consider at least the idea with the cleanup function in curl_xxx_perform. So even it is not the nicest way to go, there would be a possibility to cleanup all half-closed connections, not driven by an event, but better than nothing.

If you like one of these ideas I could propose a patch

cheers, Frank
-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-library
Etiquette: http://curl.haxx.se/mail/etiquette.html
Received on 2009-11-13