> -----Original Message-----
> From: curl-library-bounces_at_cool.haxx.se
> [mailto:curl-library-bounces_at_cool.haxx.se] On Behalf Of
> Daniel Stenberg
> Sent: Tuesday, May 04, 2010 5:04 PM
> To: libcurl development
> Subject: RE: Reflection for Secure IT Server
>
> Yes, but the sftp_write.c example is using blocking mode so
> it isn't directly comparable to what libcurl does. When doing
> things non-blocking of course there needs to be some socket
> operations involved.

I seem to have found the reason by comparing the two traces captured when using libcurl and when using sftp_write_nonblock:
========================================================
/* trace snippet with bad transfer with curl/libcurl */
...
[libssh2] 23.847598 SFTP: Data begin - Packet Length: 10
[libssh2] 23.847605 Socket: Error recving 16384 bytes to 0x88a7de8+0: 11
[libssh2] 23.847611 Conn: channel_read() got 10 of data from 0/0/0 [ul]
[libssh2] 23.847620 SFTP: Received packet 102 (len 10)
[libssh2] 23.847629 SFTP: Asking for 102 packet
[libssh2] 23.847640 SFTP: Open command successful
[libssh2] 23.847687 SFTP: Writing 16384 bytes
...
=> libssh2_transport_read() plain (67 bytes)
0000: 5F 00 00 00 00 00 00 00 01 00 00 00 36 53 46 54 : _...........6SFT
0010: 50 20 73 65 72 76 65 72 20 74 65 72 6D 69 6E 61 : P server termina
0020: 74 65 64 20 62 79 20 65 78 63 65 70 74 69 6F 6E : ted by exception
0030: 3A 20 70 61 63 6B 65 74 20 74 6F 6F 20 6C 6F 6E : : packet too lon
0040: 67 0D 0A : g..
... / all the way to failure exit */

/* trace snippet with good transfer with sftp_write_nonblock */
[libssh2] 7.757327 SFTP: Data begin - Packet Length: 10
[libssh2] 7.757335 Socket: Error recving 16384 bytes to 0x98c85e0+0: 11
[libssh2] 7.757341 Conn: channel_read() got 10 of data from 0/0/0 [ul]
[libssh2] 7.757346 SFTP: Received packet 102 (len 10)
[libssh2] 7.757351 SFTP: Asking for 102 packet
[libssh2] 7.757359 SFTP: Open command successful
libssh2_sftp_open() is done, now send data!
[libssh2] 7.769403 SFTP: Writing 1024 bytes
... /* all the way until normal shutdown */
========================================================
You will notice in every writing to the socket, curl uses a buffer of 16KB, while sftp_write_nonblock uses a buffer of 1KB. It is this difference that prompts the remote server to scream "SFTP server terminated by exception:: packet too long".

Back to "curl-7.20.1/include/curl/curl.h", the following is found:
========================================================
  /* Tests have proven that 20K is a very bad buffer size for uploads on
     Windows, while 16K for some odd reason performed a lot better.
     We do the ifndef check to allow this value to easier be changed at build
     time for those who feel adventurous. The practical minimum is about
     400 bytes since libcurl uses a buffer of this size as a scratch area
     (unrelated to network send operations). */
#define CURL_MAX_WRITE_SIZE 16384
========================================================
From the comment, it seems you have been hit by the similar problem before. The constant is tentatively changed to 8192 (half of 16KB), and the change makes the difference. The transfer to the SFTP server succeeds! So, it seems that this Reflection server has a stricter/smaller buffer size requirement.

Tomorrow, I'll tried to increase it to 12KB to see whether it still works. I want to increase the buffer size to as big a value as possible, to enhance the transfer speed.

Thanks,
Xu Qiang
-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-library
Etiquette: http://curl.haxx.se/mail/etiquette.html
Received on 2010-05-05