For my application, libcURL occasionally times out inside of
Curl_GetFTPResponse. I'm not sure where the cause of this problem lies, but
if I destroy my curl-easy handle and reconnect, things go smoothly on the
second attempt. In other words, I've got a suitable workaround and
unfortunately do not have the time to investigate this aspect deeply enough
to make the timeout go away.

However, this problem does expose a shortcoming in libcURL. I can detect
this hang by using CURLOPT_TIMEOUT, but that appears to limit the entire
transfer's time. Since my customers often be upload *very* large files
(100+ megabyte animated texture maps are not uncommon), I cannot use this
option to govern the transfer time. I had hoped that
CURLOPT_LOW_SPEED_LIMIT and CURLOPT_LOW_SPEED_TIME would detect this
problem, but this appears to not have any effect on Curl_GetFTPResponse.

From my perspective, because Curl_GetFTPResponse is really supposed to grab
one line or so of data from the socket, CURLOPT_LOW_SPEED_* are really
overkill for it anyway. What is really needed is the ability to ensure that
the FTP server is responding at all, as opposed to ensuring that the
bandwidth is available. The current default timeout (ftp->response_time) of
60 minutes is IMHO way too long. In my experience, if an FTP server doesn't
respond within 10 seconds, it's not going to respond at all.

In order to work around this shortcoming, I have implemented a new option,
CURLOPT_FTP_RESPONSE_TIMEOUT, that, when set, replaces the timeout value in
Curl_GetFTPResponse with the (long) value set with curl_easy_setopt( ...
CURLOPT_FTP_RESPONSE_TIMEOUT ... ):

    if(data->set.ftp_response_timeout )
      /* if CURLOPT_FTP_RESPONSE_TIMEOUT is set, use that to determine
         remaining time. */
      timeout = data->set.ftp_response_timeout - /* timeout time */
        Curl_tvdiff(Curl_tvnow(), conn->now)/1000; /* spent time */
    else if(data->set.timeout)
      /* if timeout is requested, find out how much remaining time we have
*/
      timeout = data->set.timeout - /* timeout time */
        Curl_tvdiff(Curl_tvnow(), conn->now)/1000; /* spent time */
    else
      /* Even without a requested timeout, we only wait response_time
         seconds for the full response to arrive before we bail out */
      timeout = ftp->response_time -
        Curl_tvdiff(Curl_tvnow(), now)/1000; /* spent time */

The number of files changed for this patch is minimal
(urldata.h,ftp.c,url.c,and curl/curl.h), and default behavior remains
completely unchanged. I would like to submit this as a patch against the
CVS version ASAP, but thought I would get some feedback before proceeding to
clean up my changes.

One issue with this version that comes to mind is the fact that
CURLOPT_TIMEOUT is effectively overriden by CURLOPT_FTP_RESPONSE_TIMEOUT for
the duration of Curl_GetFTPResponse, but since CURLOPT_FTP_RESPONSE_TIMEOUT
will likely be set to a value much smaller than CURLOPT_TIMEOUT if they are
ever both set at the same time, I think this is not likely to ever surface.
In any event, it could be corrected by doing something like this:

if ( data->set.ftp_response_timeout )
  /* use data->set.ftp_response_timeout */

if ( data->set.timeout
  && ( !data->set.ftp_response_timeout
    || data->set.timeout < data->set.ftp_response_timeout ) )
  /* use data->set.timeout */

Thoughts, suggestions?

--
-- Early Ehlinger CEO, ResPower Inc - Toll-Free : 866-737-7697
-- www.respower.com -- 500+ GHz Supercomputer Starting At USD$0.50/GHz*Hour
-------------------------------------------------------
This sf.net email is sponsored by:ThinkGeek
Welcome to geek heaven.
http://thinkgeek.com/sf