On Thu, 30 Sep 2010, Joshua Kwan wrote:

>> This is primarily due to the way the SFTP protocol works and how libssh2 is
>> made to speak that protocol. It is a known limitation of libssh2 and we've
>> talked within that project on what can be done to overcome it. Nobody has
>> yet tried to actually write any code to make it happen though.
>
> Is there a mailing list thread about that that you could link to? Seems like
> an interesting problem.

I tried to find a nice writeup of this problem but I failed (even though I'm
convinced I've written about this before on the libssh2-devel list). So, I'll
instead go slightly off-topic and describe the SFTP problem here!

SFTP is a protocol on top of SSH. When you ask for a file over SFTP, you do
that similar to how a file system works. You basically send an OPEN first and
then you send repeated READ commands over and over again until the file has
been transfered.

The problem with this is that if you send a request for data, wait for the
data to arrive fine and then send the next read request, you get a back-and-
forth effect with a round-trip and delay between each packet.

This is what libssh2 does.

To increase download speed, the openssh approach is to make sure that several
READ requests are sent away at any given time so that there is less waiting
and you instead better make sure to keep the pipe filled up. A bit similar to
HTTP pipelining.

I'm not ruling out that there are other tweaks to do to further increase SFTP
download speeds, but libssh2 is already close to openssh speedwise for SCP so
I think this should be the biggest necessary change to do.

-- 
  / daniel.haxx.se
-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-library
Etiquette:  http://curl.haxx.se/mail/etiquette.html