cURL / Mailing Lists / curl-library / Single Mail

curl-library

Re: Getting files from SFTP is slower at newer versions

From: Daniel Stenberg <daniel_at_haxx.se>
Date: Sun, 15 May 2016 11:45:39 +0200 (CEST)

On Sun, 15 May 2016, Moti Avrahami wrote:

> I recently upgraded my libcurl version from 7.19.4 to 7.47.1 and I noticed
> that there is a performance downgrade when downloading files from SFTP
> servers than the older libcurl version.
>
> I checked both versions against the same SFTP servers.
>
> I use libssh2 version 1.4.1.
>
> Does anyone know what is reason and how can I fix it?

I don't know the reason. But you're doing a *massive* upgrade bump so almost
everything has been changed internally. Clearly at some point the SFTP
performance got worse. How big is the speed difference you measure? What's the
latency to the server in your test case?

You're also using an outdated libssh2 version. It has improved through time
and modern versions should be faster and more reliable.

The way to track this down is through good old fashioned debugging. You can
possibly bisect yourself to the actual offending commit (assuming it was a
single specific one) and based on knowing which change it was we can perhaps
find the culprit/explanation faster.

I've written about (poor) SFTP performance in libcurl several times in the
past and we have ideas on how it could be increased significantly, but
unfortunately nobody has yet actually converted any of those thoughts into
reality. One such mention is here:
https://curl.haxx.se/docs/todo.html#Modified_buffer_size_approach

-- 
  / daniel.haxx.se
-------------------------------------------------------------------
List admin: https://cool.haxx.se/list/listinfo/curl-library
Etiquette:  https://curl.haxx.se/mail/etiquette.html
Received on 2016-05-15