Hi All,

I have had very good experience using libCurl for downloading files using
FTP (it downloaded files atleast 2.5 seconds faster than wget).

Now, I am trying to download files using http, to my surprise, I see libCurl
performing much slower than python's httplib, url2lib and even wget. Below
are the timing details for downloading
http://curl.haxx.se/download/curl-7.19.6.tar.gz, 50 times on Solaris
machine, timing details have been obtained using *time* command,

Python's urllib2 - real 0m27.29s, user 0m1.11s, sys 0m1.35s

Python's httplib - real 0m21.19s, user 0m0.80s, sys 0m1.23s

libCurl - real 5m18.91s, user 0m0.60s, sys 0m1.99s

curl - real 5m43.95s, user 0m0.45s, sys 0m1.93s

wget - real 0m19.25s, user 0m0.00s, sys 0m1.79s

I am aware that for general computation of speed only user time + sys time
is considered. But consistently if the application takes so much time, how
efficient and fast is it? (5 minutes and 43 seconds for libcurl comapred to
21 seconds for httplib)

Should I enable any feature for faster access? When I looked at the trace
for libcurl using truss, pollsys function appears numerous times, between
every write there are 2 pollsys fucntions seen,

4196 write(3, " lA3 1 , k u8819BBC1 888".., 8192) = 8192
4197 pollsys(0xFFBFEFD0, 1, 0xFFBFEF50, 0x00000000) = 1
4198 pollsys(0xFFBFEF10, 1, 0xFFBFEE90, 0x00000000) = 1
4199 recv(5, " CC1EBD297E2FDBAFB92 5EB".., 12510, 0) = 8088
4200 write(3, " hB7D4 O12 nF5B71A14 FB9".., 8192) = 8192

It would be great help, if anyone can let me know, if I am doing soemthing
wrong here.

Warm Regards,
Sandeep.

PS : the following statements are used for file download using libcurl,

  curl_easy_setopt(curl_handle, CURLOPT_URL, url);
  curl_easy_setopt(curl_handle, CURLOPT_NOPROGRESS, 1L);
  curl_easy_setopt(curl_handle, CURLOPT_WRITEFUNCTION, write_data);
  bodyfile = fopen(bodyfilename,"w");
  if (bodyfile == NULL) {
     curl_easy_cleanup(curl_handle);
     return -1;
  }
  curl_easy_setopt(curl_handle, CURLOPT_WRITEDATA, bodyfile);
  curl_easy_perform(curl_handle);

static size_t write_data(void *ptr, size_t size, size_t nmemb, void *stream)
{
  int written = fwrite(ptr, size, nmemb, (FILE *)stream);
  /*printf("In Write data, size = %d\n", size) ;*/
  return written;
}
Received on 2009-09-03