cURL / Mailing Lists / curl-library / Single Mail

curl-library

Re: Error when doing FTP upload via http_proxy tunnel: "Transferred a partial file"

From: Ramprasad N <ramprasad85_at_gmail.com>
Date: Wed, 9 Jul 2014 14:54:48 +0530

Thanks for your reply Ray,
Following is my build system configuration (64bit ubuntu 12.04). The
customer's configuration is also the same but just that it is behind
an unknown proxy server.

>uname -a
Linux ramprasad-VirtualBox 3.2.0-23-generic #36-Ubuntu SMP Tue Apr 10
20:39:51 UTC 2012 x86_64 x86_64 x86_64 GNU/Linux
>curl -V
curl 7.38.0-DEV (x86_64-unknown-linux-gnu) libcurl/7.38.0-DEV
OpenSSL/1.0.1 zlib/1.2.3.4 libidn/1.23 librtmp/2.3
Protocols: dict file ftp ftps gopher http https imap imaps ldap ldaps
pop3 pop3s rtmp rtsp smtp smtps telnet tftp
Features: IDN IPv6 Largefile NTLM NTLM_WB SSL libz TLS-SRP

I compile using curl-config --static-libs

The code which I had posted itself fails with "Transferred a partial
file" error.
The issue is not specific to any particular sample file.
That same code works fine in my system via squid proxy (but when I
tried FreeProxy, the FreeProxy service crashes almost every time. That
doesn't happen with filezilla! so I switched to squid)
I could not find what proxy is being used at the customer location.

Here is my observation:
When I upload a file using filezilla via http_proxy it uploads using
CONNECT and once it reaches 100% it waits for some time (probably for
the server to write the data to disc) and then shows the response from
server and then moves the transfer from queue to successful transfer
list.
When I upload the same file using the code I had posted, it also sends
the same FTP commands via proxy but the moment the read_function
returns 0, libcurl (or maybe the proxy) terminates the connection and
returns that error. The FileZilla server log says 'unable to send
reply to client'.

So I tried a dirty trick to workaround this. Instead of returning 0
where there is no more data to send, for about 40 callbacks I memset
the buffer to 0 and return nmemb*size. This also generates the same
error but only after the required amount of data + about 500KB of
zeros have been sent. I'm temporarily okay with this because when I
download I'll skip those extra 0s (i get the actual filesize from the
header). The downloading part is also written using libcurl and that
works perfectly. Only uploading is the issue.

I'll find out what proxy is being used from the customer and post it.
Will changing any of those timeouts help?

Thanks
Ramprasad

> Date: Wed, 02 Jul 2014 14:19:08 -0400
> From: Ray Satiro <raysatiro_at_yahoo.com>
> To: curl-library_at_cool.haxx.se
> Subject: Re: Error when doing FTP upload via http_proxy tunnel:
> "Transferred a partial file"
> Message-ID: <53B44D1C.6020206_at_yahoo.com>
> Content-Type: text/plain; charset=UTF-8; format=flowed
>
>
> On 6/30/2014 11:23 PM, Ramprasad N wrote:
> > Hi,
> > The uploaded file is 34600 bytes smaller
> > When I tried uploading another file it was 64488 bytes smaller
> > This happens for all files. The last few bytes never reaches the server.
> > But the same files are getting uploaded well by using FileZilla client
> >
> > This happens only when done via proxy
> > It happens irrespective of the ftp server (I've tried IIS server and
> > FileZilla server)
> >
> > libcurl is set to pickup proxy details automatically
> >
> > Here is the verbose output (I had to replace the actual server name
> > with ---myserver.com---)
> >
> [...]
> > I'm using libcurl version 7.83
> >
> > This sample code (based on
> > http://curl.haxx.se/libcurl/c/fileupload.html) specifies
> > CURLOPT_INFILESIZE_LARGE.
> > My actual code also fails with the same error, it uses the
> > CURLOPT_READFUNCTION callback to upload dynamically generated data
> > without specifying INFILESIZE.
> >
> > I've been trying since three days to get this working......
>
> I can try to reproduce this but I need a sample. You say you are based
> on the example code but you have your own readfunction. I would need a
> sample where the problem happens to try to reproduce. Also check your
> libcurl version number again and any information you can get on the
> proxy. I know there is a problem with Fiddler proxy ftp via CONNECT, but
> I did a wireshark and I think it is a Fiddler problem not a curl
> problem. That is the only one I am aware of and it doesn't look like
> you're connecting to a Fiddler proxy. Please provide the details listed
> here http://curl.haxx.se/docs/bugs.html
>
>
>
>
-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-library
Etiquette: http://curl.haxx.se/mail/etiquette.html
Received on 2014-07-09