cURL / Mailing Lists / curl-library / Single Mail

curl-library

Re: Please advise on how to improve libcurl HTTP GET performance

From: Alex <Alexxxx89_at_ya.ru>
Date: Tue, 11 Oct 2011 00:23:24 +0400
1) No way, I'm downloading to a memory buffer.
2) I didn't try telnet, but both Opera and console CURL tool download the very same page at ~300 KB/s, which is much closer to my actual net speed. Just a reminder: libcurl gives me 25-40 KB/s.
Here's the code, you can see what page I'm testing on.šCByteArray is just a wrapper for void* buffer, it's append uses realloc and memcpy.
size_t onDownloadCallback( char* ptr, size_t size, size_t nmemb, void* receiver )
{
š š size_t actualSize = size * nmemb;
š š CByteArray* dest = (CByteArray*) receiver;
š š if (!dest)
š š return 0;
š š dest->append(ptr, actualSize);
š š return actualSize;
}

void main ()
{
šCURL *curl; CURLcode res;

š š CHtmlPage page (CUrl(""));
š š curl = curl_easy_init();
š š time_t start,stop;
š š if(curl) {
š š š š curl_easy_setopt(curl, CURLOPT_WRITEDATA, &page.byteArray());
š š š š curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, onDownloadCallback);
š š š š curl_easy_setopt(curl, CURLOPT_URL, "http://www.math.utah.edu/~pa/math/largeprime.html");
šš š š šstart = clock();
š š š š res = curl_easy_perform(curl);
š š š š stop = clock();
š š š š curl_easy_cleanup(curl); š}

š š š š cout << "Download speed: " << page.byteArray().getSize() * 1000 / ((stop - start) * 1024) << " KB/s\n";
}
The code still looks perfectly OK to me, don't understand what's the bottleneck.š
š
Regards,
Alexš
10.10.2011, 20:51, "Alan Wolfe" <alan.wolfe@gmail.com>:

#1 - Does your program block at all on disk i/o (are you reading or writing to the disk?) if so that could be bottlenecking your D/L.
#2 - Are you sure the problem is on your end, not the server side?

-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-library
Etiquette: http://curl.haxx.se/mail/etiquette.html
Received on 2011-10-10