Hello,
After several days browsing this list and debugging latest cURL, I
beleive I found the bug.
Actually similar behaviour described here
http://curl.haxx.se/mail/archive-2008-07/0097.html (SFTP fail to
retrieve large file)

So that is right, some correlation beetwin big file and this failure
exists, file cant be bigger then 16384
Also you can read this:
 
http://www.mail-archive.com/libssh2-devel@lists.sourceforge.net/msg00652.html

Now what I found:

Lets start from the lower level:

Libssh2->packet.c line 677

                 if (session->packAdd_channel->remote.packet_size <
                     (datalen - session->packAdd_data_head)) {
                     /*
                      * Spec says we MAY ignore bytes sent beyond
                      * packet_size
                      */
                     libssh2_error(session,
 
LIBSSH2_ERROR_CHANNEL_PACKET_EXCEEDED,
                                   "Packet contains more data than we
offered to receive, truncating",
                                   0);
                     datalen =
                         session->packAdd_channel->remote.packet_size +
                         session->packAdd_data_head;
                 }

For SSH it is not a problem , it makes adjustment and returns an error
(not critical in case you care about it). Curl doesn’t and getting
confused with truncated data.
So what happens on curl site ? This:

#define CURL_MAX_WRITE_SIZE 16384

Then

/* Download buffer size, keep it fairly big for speed reasons */
#undef BUFSIZE
#define BUFSIZE CURL_MAX_WRITE_SIZE

#define HEADERSIZE 256

And finally:

     bytesfromsocket = MIN((long)sizerequested,
conn->data->set.buffer_size ?
                           conn->data->set.buffer_size : BUFSIZE);

if(conn->protocol & PROT_SFTP)
       nread = Curl_sftp_recv(conn, num, buffertofill, bytesfromsocket);

Ok, curl requesting buffer with magic number 16384 + “head data” and it
is always generating error LIBSSH2_ERROR_CHANNEL_PACKET_EXCEEDED for
that specific server (looks like it sends extra data which is “Spec says
we MAY ignore bytes sent beyond” – just legal, but not for curl).
Then data truncated and makes data out of sync with data len, curl
getting crazy and connection dropped by sftp server.

Simple fix is to reduce CURL_MAX_WRITE_SIZE and it works. How many bytes
? I believe it depends on server, but something like 16000 will work.
Or just disable data truncation in libssh2 which is also works. I
beleive there is better patch but I dont have time to dig cURL, so
probably developers could take care about this special case ?

Aleksey
-------------------------------------------------------------------
List admin: http://cool.haxx.se/cgi-bin/mailman/listinfo/curl-users
FAQ: http://curl.haxx.se/docs/faq.html
Etiquette: http://curl.haxx.se/mail/etiquette.html
Received on 2008-09-18