Buy commercial curl support from WolfSSL. We help you work
out your issues, debug your libcurl applications, use the API, port to new
platforms, add new features and more. With a team lead by the curl founder
himself.
ftpd on-the-fly conversions incompatible with curl
- Contemporary messages sorted: [ by date ] [ by thread ] [ by subject ] [ by author ] [ by messages with attachments ]
From: Daniel Feenberg via curl-users <curl-users_at_lists.haxx.se>
Date: Thu, 20 Apr 2023 19:03:00 -0400 (EDT)
I have been using the NetBSD ftpd server for some time specifically because of
the on-the-fly conversion capability. Usually that is for gziping files
before transmission, but I have a more critical need - I use it to process data
for anonymous users. The NetBSD ftpd configuration file allows the
available conversions to depend on additions to the filename. For example:
conversion all .gz f . /usr/bin/gzip -c -- %s
will cause a file "foo" to be sent gzipped if file "foo.gz" is requested.
This won't work with GUI ftp clients (foo.gz won't be in the directory
listing), so I would advise my users to use curl instead.
However, it seems that curl inquires about the size of the file before "get"ing
it, and ftpd replies "550 foo.gz not a plain file" when a file is subject
to conversion, which is apparently treated as a fatal error by curl, and curl
does not attempt to "get" the file. Here is a portion of the --verbose output
from curl:
> TYPE I
< 200 Type set to I.
> SIZE foo.gz
< 550 foo.gz: not a plain file.
* The file does not exist
According to this post (https://curl.se/mail/lib-2004-10/0035.html), if
the SIZE command did not exist, curl would still work. It appears that
existing and providing a specific error message is the getting in the way.
I am not sure why curl bothers to issue the SIZE command even when the
-ignore_content_length option is specified. It seems unnecessary.
Is there any way around this? There is information about curl option
ignore_content_length at https://curl.se/mail/lib-2015-09/0121.html
showing that it was added for the case of files that grew while being
downloaded, a slightly different situation. In that case the filename
requested does exist.
Daniel Feenberg
http://taxsim.nber.org/
Date: Thu, 20 Apr 2023 19:03:00 -0400 (EDT)
I have been using the NetBSD ftpd server for some time specifically because of
the on-the-fly conversion capability. Usually that is for gziping files
before transmission, but I have a more critical need - I use it to process data
for anonymous users. The NetBSD ftpd configuration file allows the
available conversions to depend on additions to the filename. For example:
conversion all .gz f . /usr/bin/gzip -c -- %s
will cause a file "foo" to be sent gzipped if file "foo.gz" is requested.
This won't work with GUI ftp clients (foo.gz won't be in the directory
listing), so I would advise my users to use curl instead.
However, it seems that curl inquires about the size of the file before "get"ing
it, and ftpd replies "550 foo.gz not a plain file" when a file is subject
to conversion, which is apparently treated as a fatal error by curl, and curl
does not attempt to "get" the file. Here is a portion of the --verbose output
from curl:
> TYPE I
< 200 Type set to I.
> SIZE foo.gz
< 550 foo.gz: not a plain file.
* The file does not exist
According to this post (https://curl.se/mail/lib-2004-10/0035.html), if
the SIZE command did not exist, curl would still work. It appears that
existing and providing a specific error message is the getting in the way.
I am not sure why curl bothers to issue the SIZE command even when the
-ignore_content_length option is specified. It seems unnecessary.
Is there any way around this? There is information about curl option
ignore_content_length at https://curl.se/mail/lib-2015-09/0121.html
showing that it was added for the case of files that grew while being
downloaded, a slightly different situation. In that case the filename
requested does exist.
Daniel Feenberg
http://taxsim.nber.org/
-- Unsubscribe: https://lists.haxx.se/mailman/listinfo/curl-users Etiquette: https://curl.se/mail/etiquette.htmlReceived on 2023-04-21