cURL / Mailing Lists / curl-library / Single Mail

curl-library

Re: Slow Upload Performance on High-Bandwidth connections on windows

From: Daniel Stenberg <daniel_at_haxx.se>
Date: Thu, 7 Feb 2013 22:10:38 +0100 (CET)

On Wed, 6 Feb 2013, Kurushin Andrey wrote:

> 1. the main bug report (filezilla vs far manager vs curl) contains test of
> cross-countries test results (ping >8ms). recent tests (with custom builds
> of 7.28.1) were made inside single subnet (ping < 1ms) so they have better
> speed values.

Well, 8 vs 1 ms latency shouldn't make much of a difference to a fairly large
plain FTP transfer.

> 2. CURL_MAX_WRITE_SIZE = 16kb and SO_SNDBUF=8mb gives very good ftp upload
> speed (equal to version with big global buffer)

Interesting! So have you tried with for example 4mb, 2mb or 512kb etc to see
if there's a particular cut-off limit somewhere?

It is a bit weird that the TCP stack can't sort this out by itself...

> but sftp upload speed still have 6Mb/s limit.

Yes, but I've explained several times on this list and on the libssh2 list,
the reason for SFTP's problems is _entirely_ different and in fact just
growing the buffer size is only one piece of the fix to the SFTP performance
bottleneck we can see.

If you really want to address the SFTP problem then I won't mind helping out,
but we cannot mix the two problems into one and pretending that they are the
same.

> So the best way to increase both ftp and sftp upload speed is to increase
> CURL_MAX_WRITE_SIZE;

Possibly. But we can't unconditionally just bump the 16K up to something
crazy like 512K. We have applications who do hundreds of even thousands of
parallel transfers, they will not appreciate such a massive memory bloat.

Possibly the time has come to start working on allowing a custom size for the
buffer.

-- 
  / daniel.haxx.se
-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-library
Etiquette:  http://curl.haxx.se/mail/etiquette.html
Received on 2013-02-07