Re: Fetching More than 500 files from Server
Date: Tue, 19 Jun 2012 09:30:14 +0530
We have added one more line to our code to reduce the connection established.
curl_easy_setopt(ptCurlHandle, CURLOPT_COOKIE, pstrSessionCookie)
After adding the CURLOPT_COOKIE, the connection issue was solved but
data fetched is zero bytes
Please let me know what may be causing this issue.
> Message: 3
> Date: Thu, 14 Jun 2012 10:33:08 +0200 (CEST)
> From: Daniel Stenberg <daniel_at_haxx.se>
> To: libcurl development <curl-library_at_cool.haxx.se>
> Subject: Re: Fetching More than 500 files from Server
> Message-ID: <alpine.DEB.2.00.1206141023270.19610_at_tvnag.unkk.fr>
> Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed
> On Thu, 14 Jun 2012, Swamy Mudhbasalar wrote:
> Please stop top-posting and full-quoting. If you intend to reply to emails
> here, switch off the digest-mode!
>> LibCurl Version : 7.21.0
> We have made 13 releases since that, so it's not unthinkable that a newer
> version works better or at least different. It would be very interesting to
> learn if you see anything similar with a recent version.
>> curl_easy_setopt(ptCurlHandle, CURLOPT_ERRORBUFFER, acErrorBuffer);
>> curl_easy_setopt(ptCurlHandle, CURLOPT_URL, pstrURL);
>> curl_easy_setopt(ptCurlHandle, CURLOPT_WRITEFUNCTION, VmwGetCB);
>> curl_easy_setopt(ptCurlHandle, CURLOPT_WRITEDATA, (void *)&tContext);
>> curl_easy_setopt(ptCurlHandle, CURLOPT_USERAGENT, pstrUserAgent);
>> curl_easy_setopt(ptCurlHandle, CURLOPT_SSL_VERIFYPEER, 0L);
>> curl_easy_setopt(ptCurlHandle, CURLOPT_SSL_VERIFYHOST, 0L);
>> curl_easy_setopt(ptCurlHandle, CURLOPT_USERPWD, pstrUserPass);
> If these are truly the only options you use then the connection: header in
> request is truly a mystery to me...
> You could consider using CURLOPT_VERBOSE to get to see about libcurl's
> to keep connections open or close them etc.
>>> In which sense is this connection then still "alive" ? How do you see it
>>> being kept in 30 minutes?
>> When I run the netstat command from the command prompt, i can see many
>> connection established. For every file there is one connection
> Even ESTABLISHED! That sounds so completely broken. When libcurl concludes
> that a connection can't be kept for further re-use it will _close_ that
> socket. If it doesn't conclude that, it will keep the connection in its
> connection pool (and possibly close an older one to make room) but that pool
> will only fit a finite number of connections and that will be much less than
> 500 by default...
>> All this connection will get closed after some time approximately 30
> So who or what is closing them after 30 minutes?
> / daniel.haxx.se
List admin: http://cool.haxx.se/list/listinfo/curl-library
Received on 2012-06-19