cURL / Mailing Lists / curl-users / Single Mail

curl-users

Wildcards in ftp file uploads

From: Michael Peterson <michaelpeterson123_at_gmail.com>
Date: Wed, 12 Jun 2013 10:44:26 -0400

I’m hoping someone can tell me more about how exactly cURL works,
specifically the “-T, --upload-file <file>” flag. I’m working on a script
to upload files automatically to a server. The desired behavior is to
upload all files of a certain type(s) to the server, ideally something like
“-T {*.zip,*.log,*.sh} ftp://1.1.1.1%94. Obviously this command doesn’t
work. I started looking into how –T would handle wildcard characters.
The only two good examples dealing with my situation seemed to be from the
curl mailing list in 2003 and 2007 stating that curl can’t/doesn’t handle
wildcards for uploads. However, this is demonstratively false now. I don’t
know when exactly it was added though, or if it's intended.

On the script I’m trying to build I’ve been able to use *.zip to upload a
single zip file. After not being able to get other extensions to work I
built a quick test server and have witnessed the following behavior. I put
3 files on the local server named test.txt test2.txt and test.zip; curl
version is 7.29.0.

curl -v ftp://1.1.1.1 –u user:Pass -T *.txt ftp://1.1.1.1
transfers only test2.txt to the remote server, first one on the file list?

curl -v ftp://1.1.1.1 –u user:Pass -T test.* ftp://1.1.1.1
transfers only test.txt to the remote server, again, first one listed? look
at ls -al order

curl -v ftp://1.1.1.1 –u user:Pass -T *.* ftp://1.1.1.1
transfers only test2.txt to the remote server, again, first file in the
list out of everything.
-created a new local file named 2.2 and tried again, this time that one got
uploaded, supporting "first file" theory

curl -v ftp://1.1.1.1 –u user:Pass -T * ftp://1.1.1.1
With the 2.2 file in place, it uploaded this. However, when I removed that
file, expecting test2.txt to be uploaded, nothing was. I believe this is
because the first thing it hit before was that file, whereas after it was
removed, the first thing it hit was a directory, tried to upload the
directory and failed. (I removed 2.2 after this)

Generated another file named test1.txt (so I now have test1.txt, test2.txt,
test.txt, and test.zip) to try the ? wildcard

curl -v ftp://1.1.1.1 –u user:Pass -T test?.txt ftp://1.1.1.1
Transfers only test1.txt, consistant with first file hypothisis

curl -v ftp://1.1.1.1 –u user:Pass -T test.??? ftp://1.1.1.1
Transfers only test.txt, again, first file listed matching those wildcards

So, what I think I've shown is that curl does support wildcards, to an
extent. I'm not sure if this is intended behavior though, or if this was
added sometime in the last ~6 years since the topic was discussed on the
mailing list.

I should note that shifting the -T to the left (curl -v -T test.txt
ftp://1.1.1.1 –u user:Pass) works for single file uploads (exact file
names), but doesn't seem to generate the same results. i.e. -T *.txt or
test.* uplodads nothing, instead of the test2.txt and test.txt respectivly.

Trying to glob the expressions i.e. {*.txt}, {test.*}, or {*.txt,*.zip}
doesn't seem to work at all, nothing is uploaded.

So, back to the topic. We know curl can upload multiple files by globbing
them together inside {}. We know that curl can handle wildcards. Can
someone tell me if this behavior is intended? and either way, how I can use
it to upload multiple files with wildcards?

-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-users
FAQ: http://curl.haxx.se/docs/faq.html
Etiquette: http://curl.haxx.se/mail/etiquette.html
Received on 2013-06-12