Could someone help me to work around an FTP related timeout issue where my
application resorts to libCurl's 1800 sec hard coded internal timeout.
The problem as shown from the problematic connection below is that for some
reason the server never saw the 0 byte ACK packet 953 and libCurl entered
the 1800 sec internal timeout waiting for the headers to arrive from the
FTP server. The second capture below can be used to contrast against the
first problematic capture.

We tried all sorts of timeout tricks in the curl_easy_setopt but I am not
sure which set of timers are relevant here.

The snippet of code below shows how I setup my default curl options - this
will end up calling CURLOPT_CONNECTTIMEOUT with a 60 second timeout.

I also tried the following settings in an attempt to create a 5 second
timeout - but all to no avail.

One trick I thought about would be to put in some sort
of CURLOPT_LOW_SPEED_LIMIT timeout prior to making the FTP connection and
cancelling this timeout in the first header or progress callback, but I am
not sure if this is a reliable means of working around this problem.

The underlying problem is that there is a very long timeout waiting for the
banner response and I don't know how to insert some timer to short circuit
this 1800 second timer

John

    // prevent banner timeouts - libCurl has a default 1 sec timeout
    // that seems to kick in after the ping has completed when we make
    // the initial connection to the FTP server. Note the timer priority
    // trick where CURLOPT_TIMEOUT is > CURLOPT_FTP_RESPONSE_TIMEOUT.
    // This is the same trick used in the progress callback.
    /*auto res = */curl_easy_setopt(pCurl, CURLOPT_TIMEOUT, 6L);
    /*auto res = */curl_easy_setopt(pCurl, CURLOPT_FTP_RESPONSE_TIMEOUT,
5L);

/**
 * Sets the curl options using the current mContextInfo.
 *
 * This never sets the URI curl field as this must be
 * done outside the context object.
 */
void
SLDBContext::setCurlOptions() {
    CURL* pCurl = mCurlHandle.get();
    // reset all curl context info
    curl_easy_reset(pCurl);
    // DEOS does not support EPSV or EPRT
    /*auto res = */curl_easy_setopt(pCurl, CURLOPT_FTP_USE_EPSV, 0L);
    /*res = */curl_easy_setopt(pCurl, CURLOPT_FTP_USE_EPRT, 0L);
    /*res = */curl_easy_setopt(pCurl, CURLOPT_NOSIGNAL, 1L);
    ///*res = */curl_easy_setopt(pCurl, CURLOPT_FTP_CREATE_MISSING_DIRS,
CURLFTP_CREATE_DIR);
#if 0
    // send out TCP keep-alive probes - not required - actually this was an
attempt at
    // performing ZeroWindowProbes to elicit a response from the DEOS
server to give us
    // the next block of data after it recovered from a zero window size
condition.
    // We ended up implementing this via sysctl:
    // 'sysctl -w net.ipv4.tcp_keepalive_intvl=5'
    // 'sysctl -w net.ipv4.tcp_keepalive_probes=100'
    // 'sysctl -w net.ipv4.tcp_keepalive_time=30'
    auto res = curl_easy_setopt(pCurl, CURLOPT_TCP_KEEPALIVE, 1L);
    // check to ensure that this is supported
    if (res == CURLE_OK) {
        // wait for at least 30 seconds before sending keep-alive probes
        // every 2 seconds
        /*res = */curl_easy_setopt(pCurl, CURLOPT_TCP_KEEPIDLE, 30L);
        /*res = */curl_easy_setopt(pCurl, CURLOPT_TCP_KEEPINTVL, 30L);
    }
#endif
    // do not perform CWD when traversing the pseudo directories
    /*res = */curl_easy_setopt(pCurl, CURLOPT_FTP_FILEMETHOD,
CURLFTPMETHOD_NOCWD);
    /*res = */curl_easy_setopt(pCurl, CURLOPT_CONNECTTIMEOUT,
getConnectTimeoutSecs());

    if (!isPASVMode()) {
        /*res = */curl_easy_setopt(pCurl, CURLOPT_FTPPORT, "-");
    }
    // used to capture header traffic
    if (mHeaderCallback) {
        /*res = */curl_easy_setopt(pCurl, CURLOPT_WRITEHEADER,
mpHeaderStream);
        /*res = */curl_easy_setopt(pCurl, CURLOPT_HEADERFUNCTION,
mHeaderCallback);
    }
    // for FTP GET operations
    if (mWriteDataCallback) {
        /*res = . */curl_easy_setopt(pCurl, CURLOPT_WRITEDATA,
&mScratchBuffer);
        /*res = */curl_easy_setopt(pCurl, CURLOPT_WRITEFUNCTION,
mWriteDataCallback);
    }
    // for FTP PUT operations
    if (mReadFileCallback) {
        /*res = */curl_easy_setopt(pCurl, CURLOPT_READFUNCTION,
mReadFileCallback);
    }

    // progress callback used to track upload progress only
    if (mProgressCallback) {
        /*res = */curl_easy_setopt(pCurl, CURLOPT_XFERINFOFUNCTION,
mProgressCallback);
        /*res = */curl_easy_setopt(pCurl, CURLOPT_XFERINFODATA, nullptr);
        /*res = */curl_easy_setopt(pCurl, CURLOPT_NOPROGRESS, 0L);
    }

    // verbose logging
    if (mDebuggingCallback) {
        /*res = */curl_easy_setopt(pCurl, CURLOPT_VERBOSE, 1L);
        /*res = */curl_easy_setopt(pCurl, CURLOPT_DEBUGFUNCTION,
mDebuggingCallback);
        /*res = */curl_easy_setopt(pCurl, CURLOPT_DEBUGDATA, nullptr);
    } else {
        /*res = */curl_easy_setopt(pCurl, CURLOPT_VERBOSE, 0L);
        /*res = */curl_easy_setopt(pCurl, CURLOPT_DEBUGDATA, nullptr);
    }

    // disable Nagle algorithm - to fix slowdown in bulk transfers
    // with large data files @JC not necessary
    //res = curl_easy_setopt(pCurl, CURLOPT_TCP_NODELAY, 1L);
    if (mSocketOptionCallback) {
        /*res = */curl_easy_setopt(pCurl, CURLOPT_SOCKOPTDATA, nullptr);
        /*res = */curl_easy_setopt(pCurl, CURLOPT_SOCKOPTFUNCTION,
mSocketOptionCallback);
    }
}

[image: Inline image 3]

[image: Inline image 2]

-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-library
Etiquette: http://curl.haxx.se/mail/etiquette.html