Buy commercial curl support from WolfSSL. We help you work
out your issues, debug your libcurl applications, use the API, port to new
platforms, add new features and more. With a team lead by the curl founder
himself.
RE: Does support for FTP growing files work in libcurl?
- Contemporary messages sorted: [ by date ] [ by thread ] [ by subject ] [ by author ] [ by messages with attachments ]
From: Robert Sumner via curl-library <curl-library_at_lists.haxx.se>
Date: Mon, 17 Oct 2022 09:45:02 +0100
-----Original Message-----
From: Daniel Stenberg <daniel_at_haxx.se>
Sent: 15 October 2022 23:29
To: Robert Sumner via curl-library <curl-library_at_lists.haxx.se>
Cc: fractal.sumner_at_btinternet.com
Subject: Re: Does support for FTP growing files work in libcurl?
On Thu, 13 Oct 2022, Robert Sumner via curl-library wrote:
> I have been trying to get libcurl FTP to work to a tailmode supporting
> FTP server with a growing file, but have not managed to get the full file
back.
...
> Am I missing something here or is it a problem?
The problem is the FTP protocol:
As the data is delivered in a separate connection and the only way to know
that the entire file has been delivered and has not been prematurely cut
off, is to verify that the final file size is indeed the same as originally
stated.
To support downloading files without knowing the exact size ahead of time
then unfortunately also equals not knowing if curl indeed got the entire
file or not.
------------
Hi Daniel,
Thanks very much for your response, and I take your point about the FTP
protocol in this respect. However, isn't the reason for having the optional
setting CURLOPT_IGNORE_CONTENT_LENGTH to give the caller a way to absolve
libcurl of providing this check and saying to it please just give me
everything the server gives you on the data socket. The onus then being on
the caller to check that they got what they wanted?
As the man page for CURLOPT_IGNORE_CONTENT_LENGTH says, this then allows
growing files to be supported, which is a nice feature, albeit one that
isn't as robust as for standard fixed transfers. The code to support this
option appears in the code base, but it looks to just have one small issue
that prevents it working. I took a clone of the latest version and put the
small fix in, which then allowed growing files to work for me locally.
Thanks
Rob
Date: Mon, 17 Oct 2022 09:45:02 +0100
-----Original Message-----
From: Daniel Stenberg <daniel_at_haxx.se>
Sent: 15 October 2022 23:29
To: Robert Sumner via curl-library <curl-library_at_lists.haxx.se>
Cc: fractal.sumner_at_btinternet.com
Subject: Re: Does support for FTP growing files work in libcurl?
On Thu, 13 Oct 2022, Robert Sumner via curl-library wrote:
> I have been trying to get libcurl FTP to work to a tailmode supporting
> FTP server with a growing file, but have not managed to get the full file
back.
...
> Am I missing something here or is it a problem?
The problem is the FTP protocol:
As the data is delivered in a separate connection and the only way to know
that the entire file has been delivered and has not been prematurely cut
off, is to verify that the final file size is indeed the same as originally
stated.
To support downloading files without knowing the exact size ahead of time
then unfortunately also equals not knowing if curl indeed got the entire
file or not.
------------
Hi Daniel,
Thanks very much for your response, and I take your point about the FTP
protocol in this respect. However, isn't the reason for having the optional
setting CURLOPT_IGNORE_CONTENT_LENGTH to give the caller a way to absolve
libcurl of providing this check and saying to it please just give me
everything the server gives you on the data socket. The onus then being on
the caller to check that they got what they wanted?
As the man page for CURLOPT_IGNORE_CONTENT_LENGTH says, this then allows
growing files to be supported, which is a nice feature, albeit one that
isn't as robust as for standard fixed transfers. The code to support this
option appears in the code base, but it looks to just have one small issue
that prevents it working. I took a clone of the latest version and put the
small fix in, which then allowed growing files to work for me locally.
Thanks
Rob
-- Unsubscribe: https://lists.haxx.se/listinfo/curl-library Etiquette: https://curl.se/mail/etiquette.htmlReceived on 2022-10-17