Download
Browse source Changelog
Documentation
Project   Bug Bounty   Help us   Known bugs   TODO Protocols   CA bundle   HTTP Cookies   HTTP/2   SSL Certs Releases   Security   Version numbers   Vulnerabilities curl tool   man page   Tutorial   HTTP scripting Who and Why
libcurl
ABI API Competitors Examples Features Mailing list Related libs Using libcurl Tutorial Testimonials
Get Help
curl-library curl-users Mailing lists Book: Everything curl Video presentations Report a bug Paid support
Development
Autobuilds Code Style Contribute Deprecate Internals Release Notes Release Procedure Roadmap Run Tests Security Process Specifications Test curl
News
Changelog Release table
curl / Mailing Lists / curl-library / Single Mail
Buy commercial curl support from WolfSSL. We help you work out your issues, debug your libcurl applications, use the API, port to new platforms, add new features and more. With a team lead by the curl founder himself.

Re: How to do optimal FTP upload for multiple files?

From: Daniel Stenberg via curl-library <curl-library_at_cool.haxx.se>
Date: Wed, 17 Jul 2019 00:50:34 +0200 (CEST)

On Tue, 16 Jul 2019, Taras Kushnir via curl-library wrote:

> The reason why I’m asking this is the following: I got reports from users
> that my app “uploads very slowly” so I started this investigation. Default
> upload code heavily (like x3-x4 on my machine) underperforms FileZilla
> upload and commandline “ftp" utility even for 1 file to 1 server (that
> eliminates my theoretical threading implementation bugs etc.).

I've not seen/heard this before.

What libcurl version on what platform? Have you seen this "slowness" on more
than one version/platform?

Is this against any or particular servers?

When you do these "slow" uploads, what's the RTT to server, roughly?

> If, when doing FTP upload for 1 file for 1 server, I’m setting
> CURLOPT_UPLOAD_BUFFERSIZE, upload speed increases for some users of my app
> to “almost satisfactory”.

When you set it to what value? What does "almost satisfactory" mean in terms
how it compares to other tools then?

> However, same setting cause FTP response timeouts on _every_ transfer for
> other users (who, if I understand correctly, have much lower bandwidth).

That sounds really weird and would rather imply a (separate) bug.

> On my machine (macbook pro 2015) and bandwidth I feel almost no difference
> when playing with UPLOAD_BUFFERSIZE.

In my own tests, UPLOAD_BUFFERSIZE mostly make a difference when you have near
infinite bandwidth, like uploading on localhost. (And when when using SFTP,
but I assume we're not talking about that here.)

> Therefore comes a question: is there any idiomatic way of using libcurl for
> such cases that will solve upload speed issues for users both with big and
> small bandwidth?

Increasing buffer size shouldn't hurt transfer speeds for any users! 

-- 
  / daniel.haxx.se | Get the best commercial curl support there is - from me
                   | Private help, bug fixes, support, ports, new features
                   | https://www.wolfssl.com/contact/

-------------------------------------------------------------------
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library
Etiquette: https://curl.haxx.se/mail/etiquette.html
Received on 2019-07-17