Re: FTP of growing files?
Date: Mon, 28 Sep 2015 12:54:24 -0700
Documentation patches. I don't know how to generate the documentation
package so I took my best guess at formatting codes.
--- curl_easy_setopt.3 2015-09-23 17:38:17.045174971 -0700
+++ curl_easy_setopt.3.new 2015-09-28 10:46:38.217736804 -0700
@@ -340,6 +340,8 @@
Send ACCT command. See \fICURLOPT_FTP_ACCOUNT(3)\fP
Specify how to reach files. See \fICURLOPT_FTP_FILEMETHOD(3)\fP
+Ignore Content-Length. See \fICURLOPT_IGNORE_CONTENT_LENGTH(3)\fP
.SH RTSP OPTIONS
RTSP request. See \fICURLOPT_RTSP_REQUEST(3)\fP
--- CURLOPT_IGNORE_CONTENT_LENGTH.3 2015-09-23 17:38:17.061174459 -0700
+++ CURLOPT_IGNORE_CONTENT_LENGTH.3.new 2015-09-28 11:48:54.391092107
@@ -22,7 +22,7 @@
.TH CURLOPT_IGNORE_CONTENT_LENGTH 3 "19 Jun 2014" "libcurl 7.37.0"
-CURLOPT_IGNORE_CONTENT_LENGTH \- ignore Content-Length in HTTP response
+CURLOPT_IGNORE_CONTENT_LENGTH \- ignore Content-Length in HTTP response
and file size in FTP request
@@ -30,18 +30,32 @@
CURLcode curl_easy_setopt(CURL *handle, CURLOPT_IGNORE_CONTENT_LENGTH,
-If \fIignore\fP is set to 1, ignore the Content-Length header in the HTTP
+This option affects both HTTP and FTP transfers. If \fIignore\fP is set
to 1, it
+has the following affects:
+Ignore the Content-Length header in the HTTP
response. This is useful for Apache 1.x (and similar servers) which will
report incorrect content length for files over 2 gigabytes. If this option
used, curl will not be able to accurately report progress, and will simply
stop the download when the server ends the connection.
+Do not request the file size from the FTP server when downloading files.
+the FTP client will terminate the transfer when it has received the number
+specified by the response to the FTP SIZE request. With an unknown file
+download will continue until the \fIserver\fP terminates the download.
+support on the server to determine when the entire file has been
Only use this option if strictly necessary.
CURL *curl = curl_easy_init();
@@ -53,6 +67,22 @@
+CURL *curl = curl_easy_init();
+ curl_easy_setopt(curl, CURLOPT_URL, "
+ /* this is a growing file on server (active transfer) so let server
+ the entire file has been transferred */
+ curl_easy_setopt(curl, CURLOPT_IGNORE_CONTENT_LENGTH, 1L);
Added in 7.14.1
.SH RETURN VALUE
I do not have a formal test case to submit, but I can state that this
modification has been applied to our product since March 2015 and our QA
department has validated it. As I mentioned, it does require support on
the FTP server to close the connection after it determines the entire file
has been transferred.
The test case would be something like this:
1) Begin transfer (upload) of a LARGE file to the FTP server.
2) While the file is being uploaded, start a download of that file from the
FTP server. Data should continue to be received as it becomes available on
3) Wait for the upload to complete.
4) Wait for the download to complete.
5) Verify that the entire file was downloaded.
On Sat, Sep 26, 2015 at 2:33 PM, Daniel Stenberg <daniel_at_haxx.se> wrote:
> On Thu, 24 Sep 2015, Kurt Fankhauser wrote:
> Here is the patch that allows setting CURLOPT_IGNORE_CONTENT_LENGTH to
>> prevent curl from requesting the file size from the ftp server. The result
>> is the connection is kept open until the server closes it when the file is
>> completely transferred.
> Ah, clever! Can you also updated the documentation to mention this? Did
> you try to write up a test case that verifies it works?
> / daniel.haxx.se
> List admin: http://cool.haxx.se/list/listinfo/curl-library
> Etiquette: http://curl.haxx.se/mail/etiquette.html
List admin: http://cool.haxx.se/list/listinfo/curl-library
Received on 2015-09-28