curl-library
Problem using Curl multi interface and persistent connections
Date: Wed, 2 Jun 2010 05:10:21 -0700
Hi,
I am facing a problem using the multi interface of curl while using
persistent connections. May be there is some lack in my understanding of
how to use curl library. But I am unable to get help from present
available documentation for what I am trying to do. I am using libcurl
version 7.19.0
The below is description of what I am doing :-
I have written an application which is receiving a number of http
requests from clients. While processing the "first" request I do the
following :-
1) Create a multi handle
2) Set the MAXCONNECTS option on this multi-handle to say 20
3) Now I create 'X' easy handles(X<=20), set their options i.e URL,
connection timeout, tranfer timeout etc and for debugging set the
verbose option as well. Also I store each of these 'X' easy handles in
data structure where I could look up an easy handle using a key. Say the
key for easy handle 1 is URL1 for easy handle2 is URL2 etc and so on ...
key for easy handleX is URLX
4) Add these 'X' easy handles to the multi-handle created in step 1
5) Do the curl_multi_perform. Get the transfer using select. All done.
6) Now I remove the 'X' easy handle from the multi handle but I do not
call curl_easy_cleanup for these 'X' easy handles.
Here is pseudo-code:-
for each of the 'X' easy handles do the following :-
curl_multi_remove_handle(multi_handle, easy_handles[i]);
//curl_easy_cleanup(easy_handles[i])); I DON'T CALL THIS
// And finally call this
curl_multi_cleanup(curl_multi_handle);
******END OF REQUEST ************
Now I have 2 questions :-
1) The major question is :- For the next request if I repeat the above
sequence of steps(X can vary in each request), except that I don't
create easy handles but look them up using the keys) and then add these
"looked up" easy handles to a "new" multi-handle, will the connections
opened in these easy handles during the first request would still be
re-used when I do curl_multi_perform() for subsequent requests(
REMEMBFER THAT I NEVER did curl_easy_cleanup() for those 'X' easy
handles and somehow i can search those handles during subsequent
requests )?
I am doing the above and with verbose option on the easy handle enabled
and found out that connect() is being called again and again with my
machine ending up in a hell lot of TIME_WAIT connections on load and
request being timed out.
So why are the present connections not being re-used.
Am I missing something in the way I am using libcurl?
Regards
Kartik Mahajan
-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-library
Etiquette: http://curl.haxx.se/mail/etiquette.html
Received on 2010-06-02