curl-library
Performance problem
Date: Fri, 21 Nov 2003 09:02:03 +0100
Hello,
I'm using libcurl 7.10.7 and I have a performance problem.
I want to send 100 files, 5 files a second which should take about 20
seconds time. The server is fast enough (tested with command line
'ftp'). Each file has a size of appr. 70kB. The output below shows the
first part of the libcurl log. At the lines where I inserted 'BREAK' the
output is delayed for about 4.5 seconds. What can I do? The same code
also works faster on other servers.
I add my code and the output. Method 'open' is called once, method
'send' is called once for each file.
Best regards
Stefan
The methods:
bool FtpPort::open(std::string& rRemoteAddress, std::string& rUserName,
std::string& rPassWord)
{
mCurl = curl_easy_init();
if( mCurl ) {
// enable uploading
curl_easy_setopt(mCurl, CURLOPT_ERRORBUFFER, arErrorMessage);
curl_easy_setopt(mCurl, CURLOPT_FTP_USE_EPSV, 0l);
// specify target URL
mTargetURL = "ftp://" + rRemoteAddress;
std::cout << "mTargetURL: " << mTargetURL << std::endl;
curl_easy_setopt(mCurl, CURLOPT_URL, mTargetURL.c_str());
// Set user and pass
mUserPass = (rUserName + ":" + rPassWord);
std::cout << "mUserPass: " << mUserPass << std::endl;
curl_easy_setopt(mCurl, CURLOPT_USERPWD, mUserPass.c_str());
// Switch on full protocol/debug output
curl_easy_setopt(mCurl, CURLOPT_VERBOSE, TRUE);
CURLcode result = curl_easy_perform( mCurl );
if (result) {
std::cout << "FtpPort::open: curl error: " << arErrorMessage
<< std::endl;
return false;
}
std::cout << "FtpPort::open: done" << std::endl;
}
else {
return false;
}
return true;
}
void FtpPort::send(std::string& rUploadFile)
{
//std::cout << "FtpPort::send(" << rUploadFile << ")" << std::endl;
const char* fileName = rUploadFile.c_str();
FILE* hd_src = fopen(rUploadFile.c_str(), "rb");
std::string renameFrom = ("RNFR (nil)");
std::string renameTo = ("RNTO " +
rUploadFile.substr(rUploadFile.find_last_of("\\") + 1));
// get the file size of the local file
struct stat fileInfo;
stat(rUploadFile.c_str(),&fileInfo);
// prepare for an upload
curl_easy_setopt(mCurl, CURLOPT_UPLOAD, TRUE);
// build a list of commands to pass to libcurl
struct curl_slist* headerlist = NULL;
headerlist = curl_slist_append(headerlist, renameFrom.c_str());
headerlist = curl_slist_append(headerlist, renameTo.c_str());
// pass in that last of FTP commands to run after the transfer
curl_easy_setopt(mCurl, CURLOPT_POSTQUOTE, headerlist);
// now specify which file to upload
curl_easy_setopt(mCurl, CURLOPT_INFILE, hd_src);
// and give the size of the upload (optional)
curl_easy_setopt(mCurl, CURLOPT_INFILESIZE, fileInfo.st_size);
// READFUNCTION must be given when INFILE is used
curl_easy_setopt(mCurl, CURLOPT_READFUNCTION, readFunction);
CURLcode result = curl_easy_perform( mCurl );
if (result) {
std::cout << "FtpPort::send: curl error: " << arErrorMessage <<
std::endl;
}
// clean up headerlist
curl_slist_free_all( headerlist );
fclose(hd_src);
}
The output:
* About to connect() to 194.95.172.11:21
* Connected to 194.95.172.11 (194.95.172.11) port 21
< 220 www.first.fraunhofer.de FTP server (Version 6.00LS) ready.
> USER bla
< 331 Password required for bla.
> PASS something
< 230 User bla logged in.
* We have successfully logged in
> PWD
< 257 "/usr2/lumos" is current directory.
* Entry path is '/usr2/lumos'
> PASV
< 227 Entering Passive Mode (194,95,172,11,197,84)
* About to connect() to 194.95.172.11:50516
* Connecting to 194.95.172.11 (194.95.172.11) port 50516
* Connected the data stream with PASV!
> TYPE A
< 200 Type set to A.
> LIST
< 150 Opening ASCII mode data connection for '/bin/ls'.
* Getting file with size: -1
total 6900
-rw-r--r-- 1 lumos guest 70310 Nov 20 18:02
lumos_out20031103150000187.bmp
[...]
-rw-r--r-- 1 lumos guest 70310 Nov 20 17:56
lumos_out20031103150019968.bmp
< 226 Transfer complete.
* Connection #0 left intact
FtpPort::open: done
* Re-using existing connection! (#0)
* Connected to (194.95.172.11) port 21
> CWD /usr2/lumos
< 250 CWD command successful.
> PASV
< 227 Entering Passive Mode (194,95,172,11,197,85)
* About to connect() to 194.95.172.11:50517
BREAK
* Connecting to 194.95.172.11 (194.95.172.11) port 50517
* Connected the data stream with PASV!
> TYPE I
< 200 Type set to I.
> STOR (nil)
< 150 Opening BINARY mode data connection for '(nil)'.
< 226 Transfer complete.
> RNFR (nil)
< 350 File exists, ready for destination name
> RNTO lumos_003185.bmp
< 250 RNTO command successful.
* Connection #0 left intact
* Re-using existing connection! (#0)
* Connected to (194.95.172.11) port 21
> CWD /usr2/lumos
< 250 CWD command successful.
> PASV
< 227 Entering Passive Mode (194,95,172,11,197,86)
* About to connect() to 194.95.172.11:50518
BREAK
* Connecting to 194.95.172.11 (194.95.172.11) port 50518
* Connected the data stream with PASV!
> TYPE I
< 200 Type set to I.
> STOR (nil)
< 150 Opening BINARY mode data connection for '(nil)'.
< 226 Transfer complete.
> RNFR (nil)
< 350 File exists, ready for destination name
> RNTO lumos_003186.bmp
< 250 RNTO command successful.
* Connection #0 left intact
* Re-using existing connection! (#0)
* Connected to (194.95.172.11) port 21
> CWD /usr2/lumos
< 250 CWD command successful.
> PASV
< 227 Entering Passive Mode (194,95,172,11,197,87)
* About to connect() to 194.95.172.11:50519
BREAK
and so on...
-------------------------------------------------------
This SF.net email is sponsored by: SF.net Giveback Program.
Does SourceForge.net help you be more productive? Does it
help you create better code? SHARE THE LOVE, and help us help
YOU! Click Here: http://sourceforge.net/donate/
Received on 2003-11-21