cURL / Mailing Lists / curl-library / Single Mail

curl-library

Re: Getting files from SFTP is slower at newer versions

From: Moti Avrahami <moti.dp_at_gmail.com>
Date: Sun, 15 May 2016 15:27:22 +0300

I measure a downgrade of more 35% download time.
I tried also to enlarge the CURL_MAX_WRITE_SIZE to 256K - 4MB but it
doesn't improve the times.
I am trying now to upgrade libssh2 to 1.7.0. Although I encounter some
compilation errors (is there a specific Visual Studio version that is
recommended for it?) I feel it will take me less time than trying to find
the problem during debugging.

Thanks,

On Sun, May 15, 2016 at 12:45 PM, Daniel Stenberg <daniel_at_haxx.se> wrote:

> On Sun, 15 May 2016, Moti Avrahami wrote:
>
> I recently upgraded my libcurl version from 7.19.4 to 7.47.1 and I noticed
>> that there is a performance downgrade when downloading files from SFTP
>> servers than the older libcurl version.
>>
>> I checked both versions against the same SFTP servers.
>>
>> I use libssh2 version 1.4.1.
>>
>> Does anyone know what is reason and how can I fix it?
>>
>
> I don't know the reason. But you're doing a *massive* upgrade bump so
> almost everything has been changed internally. Clearly at some point the
> SFTP performance got worse. How big is the speed difference you measure?
> What's the latency to the server in your test case?
>
> You're also using an outdated libssh2 version. It has improved through
> time and modern versions should be faster and more reliable.
>
> The way to track this down is through good old fashioned debugging. You
> can possibly bisect yourself to the actual offending commit (assuming it
> was a single specific one) and based on knowing which change it was we can
> perhaps find the culprit/explanation faster.
>
> I've written about (poor) SFTP performance in libcurl several times in the
> past and we have ideas on how it could be increased significantly, but
> unfortunately nobody has yet actually converted any of those thoughts into
> reality. One such mention is here:
> https://curl.haxx.se/docs/todo.html#Modified_buffer_size_approach
>
> --
>
> / daniel.haxx.se
> -------------------------------------------------------------------
> List admin: https://cool.haxx.se/list/listinfo/curl-library
> Etiquette: https://curl.haxx.se/mail/etiquette.html

-------------------------------------------------------------------
List admin: https://cool.haxx.se/list/listinfo/curl-library
Etiquette: https://curl.haxx.se/mail/etiquette.html
Received on 2016-05-15