curl / Mailing Lists / curl-library / Single Mail
Buy commercial curl support from WolfSSL. We help you work out your issues, debug your libcurl applications, use the API, port to new platforms, add new features and more. With a team lead by the curl founder himself.

Re: How to do optimal FTP upload for multiple files?

From: Daniel Stenberg via curl-library <curl-library_at_cool.haxx.se>
Date: Fri, 19 Jul 2019 16:22:29 +0200 (CEST)

On Fri, 19 Jul 2019, Daniel Jeliński wrote:

> Here's what we found in the logs:
>
> 18:54:41.002 T#12216 Connectivity::my_trace - "== Info: We are
> completely uploaded and fine"
> 18:54:41.002 T#12216 Connectivity::my_trace - "== Info: Remembering we
> are in dir \"\""
> 18:54:51.012 T#12216 Connectivity::my_trace - "== Info: FTP response timeout"
> 18:54:51.012 T#12216 Connectivity::my_trace - "== Info: control
> connection looks dead"

This does indeed look like the control connection has been dropped while the
transfer was done. At least that's the symptom of that.

> The problem is that the first line above (completely uploaded and fine) is
> logged when the OS accepts the last application buffer into OS buffer. And
> Windows accepts buffers whole - send never returns a partial result, it's
> either all or nothing. So we log that we are finished while we still have
> 1MB outstanding on data connection.

That would be highly annoying and special, and this winsock docs[1] says
otherwise:

   On nonblocking stream oriented sockets, the number of bytes written can be
   between 1 and the requested length, depending on buffer availability on both
   the client and server computers.

[1] =
https://docs.microsoft.com/en-us/windows/win32/api/winsock2/nf-winsock2-send

> Curl FTP expects a response on control connection within 10 seconds
> after it sends the last data buffer, and then declares the connection
> dead. While we could probably modify that timeout, we have no way to
> tell how much time is enough.

Right, but longer than 10 seconds response time in a case where the control
connection actually is still alive should be extremely rare.

-- 
  / daniel.haxx.se | Get the best commercial curl support there is - from me
                   | Private help, bug fixes, support, ports, new features
                   | https://www.wolfssl.com/contact/

-------------------------------------------------------------------
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library
Etiquette: https://curl.haxx.se/mail/etiquette.html
Received on 2019-07-19