cURL / Mailing Lists / curl-library / Single Mail

curl-library

Re: "100% CPU usage during SFTP transfer" bugfix

From: Daniel Stenberg <daniel_at_haxx.se>
Date: Tue, 18 Aug 2009 11:05:09 +0200 (CEST)

On Tue, 18 Aug 2009, Vourhey wrote:

> I use lubcurl 7.19.5 with libssh2-1.2 on Linux. I can see 100% CPU usage
> when I try to upload any file by SFTP. This bug is already described on the
> tracker:
> http://sourceforge.net/tracker/?func=detail&atid=100976&aid=2816745&group_id=976

Is it really? That bug was unclearly described and the reporter never came
back to help us sort it out.

For example, he was using an older libssh2 than you. Are you also on Windows?

> This bug is due to using non-blocking mode enabled by default.

No, that's not the reason. libcurl should use libssh2 non-blockingly, so the
bug may be related to that fact but it's not the reason itself.

It does give me the feeling it might be a libssh2 bug, but without more
details we can't tell for sure.

> Infinite loop (invoking of clock_gettime, poll, recv functions) loads
> CPU.

But why does it loop? When the EAGAIN situation is returned from libssh2,
libcurl checks for what to wait for socket-wise and that shouldn't happen
immediately. Is libssh2 telling us to wait for the wrong direction in this
situation? Can you dig up more specific details on what exactly is happening?

> So, I've change this code to enable blocking mode for the libssh2:
>
>     libssh2_session_set_blocking(sshc->ssh_session, 1);
>
> After libcurl recompiling, file uploading by SFTP loads CPU to ~9%. And
> looks like everything is ok: downloading/uploading files by SFTP work fine.

That's weird. libcurl is supposed to do that blocking magic on its own using
the same mechanism that libssh2 uses internally when blocking is selected. It
would then rather indicate that libssh2 _is_ right and that libcurl gets
confused for some reason.

> Could you please tell me, can these changes cause any issues in libcurl
> work?

Yes. It'll make libcurl work less like it is supposed to, as libcurl wants to
work non-blocking with the transfer-layer. It'll be most evident if you use
the multi interface.

> Are there any plans to submit fix for this bug in the next version of
> libcurl?

I'd like us to fix all bugs, yes. But fixing bugs is a lot of hard work and
we're all busy people. I hope we can help you track this down when you tell us
more details from what you experience.

> Can the fix described above be included?

No, that's not a fix - it's a work-around that you can use in your local build
but it's not suitable for the mainline code.

-- 
  / daniel.haxx.se
Received on 2009-08-18