cURL / Mailing Lists / curl-library / Single Mail

curl-library

Re: FTP of growing files?

From: Daniel Stenberg <daniel_at_haxx.se>
Date: Mon, 2 Mar 2015 08:16:14 +0100 (CET)

On Sun, 1 Mar 2015, Kurt Fankhauser wrote:

>> If the server wanted to, it could keep the connection alive and send data
>> as it arrives but I've never seen such an implementation - but even if it
>> did, it would make the final size differ from the size the file claimed to
>> be at the beginning which would cause libcurl to return an error due to
>> that
>
> Well, from the log messages that is not what I observe. The error message
> appears to be originating from the server (and reported by libcurl) due the
> the connection being closed rather than libcurl complaining about too many
> bytes received. Maybe both things are happening?

Ah yes, I believe I was slightly wrong. If the server has given a size of the
file prior to the download, then libcurl will only download exactly that
amount. If the server didn't provide a size, libcurl will assume that the
closing of the connection means the end of file is reached. The downside with
this latter method is of course that libcurl cannot tell if the connection
ends prematurely - which happens from time to time.

Meaning, the second method _could_ be used to serve a file while it is growing
on the server side.

> Okay, a couple questions since I definitely need this functionality:
> 1) Long term, is this a feature that could eventually be added to libcurl?

It depends on what exactly it means, protocol and technical wise.

> 2) Short term, do you have a gut feeling for how difficult it would be to
> add this functionality? This is something I may have to do for our system,
> but I have not yet looked at source code.

If you're just thinking of not sending a SIZE response and then just keep the
connection alive while the file is growing, it would depende entirely on the
internals of the FTP server and what knowledge it has about currently uploaded
files from clients. I guess it could be a decently small change.

A much simpler solution is to wait until the upload is complete and then
rename the file to the final one and only then will the client detect it and
download... =)

-- 
  / daniel.haxx.se
-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-library
Etiquette:  http://curl.haxx.se/mail/etiquette.html
Received on 2015-03-02