cURL / Mailing Lists / curl-library / Single Mail

curl-library

Re: "100% CPU usage during SFTP transfer" bugfix

From: Daniel Stenberg <daniel_at_haxx.se>
Date: Fri, 21 Aug 2009 00:25:54 +0200 (CEST)

On Thu, 20 Aug 2009, Kamil Dudka wrote:

> I don't think it's a libssh2 bug. I've debugged it for a while and it seems
> libssh2 is returning correct transfer direction info. However curl is not
> willing to use the value.
>
> libcurl saves the direction properly to conn->proto.sshc.waitfor, but the
> variable is not accessed during transfer at all. The common layer
> (transfer.c) then expects an outgoing transfer during upload while libssh2
> is waiting for ACK (incoming transfer).

Really? What about ssh_block2waitfor() and ssh_perform_getsock() in lib/ssh.c?

ssh_block2waitfor() is supposed to figure out the reason for a "block"
(EAGAIN) and store that as a waitfor bitmask, and then ssh_perform_getsock()
uses that for the multi interface.

For the easy interface in ssh_easy_statemach(), libcurl uses
libssh2_session_block_directions() directly to figure out what to wait for.
Quite possibly that should rather use the 'waitfor' variable instead. Can you
compare the results from that libssh2_session_block_directions() call with the
contents of the waitfor variable to see if you can detect deviances during
transfer?

> The attached patch fixes the problem, but I am not going to apply it as it's
> more likely a workaround than a fix.

Right, since it will make the multi interface less non-blocking it isn't a
suitable.

> Note that it doesn't loop with SCP upload, but only with SFTP upload for me.

Not too surprising really, SFTP is doing a lot more back-and-forth sending of
packets during a transfer than SCP so SFTP is more likely to end up in a
situation where it blocks in the "reverse" direction (like you're downloading
but get EAGAIN on the send() or vice versa).

-- 
  / daniel.haxx.se
Received on 2009-08-21