cURL / Mailing Lists / curl-library / Single Mail

curl-library

Performance problem

From: Stefan Sassenberg <stefan.sassenberg_at_gmx.de>
Date: Fri, 21 Nov 2003 09:02:03 +0100

Hello,

I'm using libcurl 7.10.7 and I have a performance problem.

I want to send 100 files, 5 files a second which should take about 20
seconds time. The server is fast enough (tested with command line
'ftp'). Each file has a size of appr. 70kB. The output below shows the
first part of the libcurl log. At the lines where I inserted 'BREAK' the
output is delayed for about 4.5 seconds. What can I do? The same code
also works faster on other servers.

I add my code and the output. Method 'open' is called once, method
'send' is called once for each file.

Best regards

Stefan

The methods:

bool FtpPort::open(std::string& rRemoteAddress, std::string& rUserName,
std::string& rPassWord)
{
    mCurl = curl_easy_init();
    if( mCurl ) {
        // enable uploading
        curl_easy_setopt(mCurl, CURLOPT_ERRORBUFFER, arErrorMessage);
        curl_easy_setopt(mCurl, CURLOPT_FTP_USE_EPSV, 0l);

        // specify target URL
        mTargetURL = "ftp://" + rRemoteAddress;
        std::cout << "mTargetURL: " << mTargetURL << std::endl;
        curl_easy_setopt(mCurl, CURLOPT_URL, mTargetURL.c_str());

        // Set user and pass
        mUserPass = (rUserName + ":" + rPassWord);
        std::cout << "mUserPass: " << mUserPass << std::endl;
        curl_easy_setopt(mCurl, CURLOPT_USERPWD, mUserPass.c_str());

        // Switch on full protocol/debug output
        curl_easy_setopt(mCurl, CURLOPT_VERBOSE, TRUE);

        CURLcode result = curl_easy_perform( mCurl );
        if (result) {
            std::cout << "FtpPort::open: curl error: " << arErrorMessage
<< std::endl;
            return false;
        }

        std::cout << "FtpPort::open: done" << std::endl;
    }
    else {
        return false;
    }

    return true;
}

void FtpPort::send(std::string& rUploadFile)
{
    //std::cout << "FtpPort::send(" << rUploadFile << ")" << std::endl;
    const char* fileName = rUploadFile.c_str();
    FILE* hd_src = fopen(rUploadFile.c_str(), "rb");
   
    std::string renameFrom = ("RNFR (nil)");
    std::string renameTo = ("RNTO " +
rUploadFile.substr(rUploadFile.find_last_of("\\") + 1));

    // get the file size of the local file
    struct stat fileInfo;
    stat(rUploadFile.c_str(),&fileInfo);

    // prepare for an upload
    curl_easy_setopt(mCurl, CURLOPT_UPLOAD, TRUE);

    // build a list of commands to pass to libcurl
    struct curl_slist* headerlist = NULL;
    headerlist = curl_slist_append(headerlist, renameFrom.c_str());
    headerlist = curl_slist_append(headerlist, renameTo.c_str());
    // pass in that last of FTP commands to run after the transfer
    curl_easy_setopt(mCurl, CURLOPT_POSTQUOTE, headerlist);

    // now specify which file to upload
        curl_easy_setopt(mCurl, CURLOPT_INFILE, hd_src);

    // and give the size of the upload (optional)
        curl_easy_setopt(mCurl, CURLOPT_INFILESIZE, fileInfo.st_size);

    // READFUNCTION must be given when INFILE is used
    curl_easy_setopt(mCurl, CURLOPT_READFUNCTION, readFunction);

    CURLcode result = curl_easy_perform( mCurl );
    if (result) {
        std::cout << "FtpPort::send: curl error: " << arErrorMessage <<
std::endl;
    }

    // clean up headerlist
    curl_slist_free_all( headerlist );

    fclose(hd_src);
}

The output:

* About to connect() to 194.95.172.11:21
* Connected to 194.95.172.11 (194.95.172.11) port 21
< 220 www.first.fraunhofer.de FTP server (Version 6.00LS) ready.
> USER bla
< 331 Password required for bla.
> PASS something
< 230 User bla logged in.
* We have successfully logged in
> PWD
< 257 "/usr2/lumos" is current directory.
* Entry path is '/usr2/lumos'
> PASV
< 227 Entering Passive Mode (194,95,172,11,197,84)
* About to connect() to 194.95.172.11:50516
* Connecting to 194.95.172.11 (194.95.172.11) port 50516
* Connected the data stream with PASV!
> TYPE A
< 200 Type set to A.
> LIST
< 150 Opening ASCII mode data connection for '/bin/ls'.
* Getting file with size: -1
total 6900
-rw-r--r-- 1 lumos guest 70310 Nov 20 18:02
lumos_out20031103150000187.bmp
[...]
-rw-r--r-- 1 lumos guest 70310 Nov 20 17:56
lumos_out20031103150019968.bmp
< 226 Transfer complete.
* Connection #0 left intact
FtpPort::open: done
* Re-using existing connection! (#0)
* Connected to (194.95.172.11) port 21
> CWD /usr2/lumos
< 250 CWD command successful.
> PASV
< 227 Entering Passive Mode (194,95,172,11,197,85)
* About to connect() to 194.95.172.11:50517
BREAK
* Connecting to 194.95.172.11 (194.95.172.11) port 50517
* Connected the data stream with PASV!
> TYPE I
< 200 Type set to I.
> STOR (nil)
< 150 Opening BINARY mode data connection for '(nil)'.
< 226 Transfer complete.
> RNFR (nil)
< 350 File exists, ready for destination name
> RNTO lumos_003185.bmp
< 250 RNTO command successful.
* Connection #0 left intact
* Re-using existing connection! (#0)
* Connected to (194.95.172.11) port 21
> CWD /usr2/lumos
< 250 CWD command successful.
> PASV
< 227 Entering Passive Mode (194,95,172,11,197,86)
* About to connect() to 194.95.172.11:50518
BREAK
* Connecting to 194.95.172.11 (194.95.172.11) port 50518
* Connected the data stream with PASV!
> TYPE I
< 200 Type set to I.
> STOR (nil)
< 150 Opening BINARY mode data connection for '(nil)'.
< 226 Transfer complete.
> RNFR (nil)
< 350 File exists, ready for destination name
> RNTO lumos_003186.bmp
< 250 RNTO command successful.
* Connection #0 left intact
* Re-using existing connection! (#0)
* Connected to (194.95.172.11) port 21
> CWD /usr2/lumos
< 250 CWD command successful.
> PASV
< 227 Entering Passive Mode (194,95,172,11,197,87)
* About to connect() to 194.95.172.11:50519
BREAK

and so on...

-------------------------------------------------------
This SF.net email is sponsored by: SF.net Giveback Program.
Does SourceForge.net help you be more productive? Does it
help you create better code? SHARE THE LOVE, and help us help
YOU! Click Here: http://sourceforge.net/donate/
Received on 2003-11-21