cURL / Mailing Lists / curl-library / Single Mail

curl-library

Problem using Curl multi interface and persistent connections

From: Kartik Mahajan <kartik.mahajan_at_pubmatic.com>
Date: Wed, 2 Jun 2010 05:10:21 -0700

Hi,

 

I am facing a problem using the multi interface of curl while using
persistent connections. May be there is some lack in my understanding of
how to use curl library. But I am unable to get help from present
available documentation for what I am trying to do. I am using libcurl
version 7.19.0

 

The below is description of what I am doing :-

 

I have written an application which is receiving a number of http
requests from clients. While processing the "first" request I do the
following :-

 

1) Create a multi handle

 

2) Set the MAXCONNECTS option on this multi-handle to say 20

 

3) Now I create 'X' easy handles(X<=20), set their options i.e URL,
connection timeout, tranfer timeout etc and for debugging set the
verbose option as well. Also I store each of these 'X' easy handles in
data structure where I could look up an easy handle using a key. Say the
key for easy handle 1 is URL1 for easy handle2 is URL2 etc and so on ...
key for easy handleX is URLX

 

4) Add these 'X' easy handles to the multi-handle created in step 1

 

5) Do the curl_multi_perform. Get the transfer using select. All done.

 

6) Now I remove the 'X' easy handle from the multi handle but I do not
call curl_easy_cleanup for these 'X' easy handles.

Here is pseudo-code:-

 

for each of the 'X' easy handles do the following :-

                curl_multi_remove_handle(multi_handle, easy_handles[i]);

                //curl_easy_cleanup(easy_handles[i])); I DON'T CALL THIS

 

// And finally call this

curl_multi_cleanup(curl_multi_handle);

 

 

******END OF REQUEST ************

 

Now I have 2 questions :-

 

1) The major question is :- For the next request if I repeat the above
sequence of steps(X can vary in each request), except that I don't
create easy handles but look them up using the keys) and then add these
"looked up" easy handles to a "new" multi-handle, will the connections
opened in these easy handles during the first request would still be
re-used when I do curl_multi_perform() for subsequent requests(
REMEMBFER THAT I NEVER did curl_easy_cleanup() for those 'X' easy
handles and somehow i can search those handles during subsequent
requests )?

 

I am doing the above and with verbose option on the easy handle enabled
and found out that connect() is being called again and again with my
machine ending up in a hell lot of TIME_WAIT connections on load and
request being timed out.

 

So why are the present connections not being re-used.

 

Am I missing something in the way I am using libcurl?

 

 

Regards

Kartik Mahajan

-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-library
Etiquette: http://curl.haxx.se/mail/etiquette.html
Received on 2010-06-02