cURL / Mailing Lists / curl-users / Single Mail

curl-users

How to make curl's glob feature place a banner before each download?

From: Kelly Jones <kelly.terry.jones_at_gmail.com>
Date: Sat, 30 May 2009 10:08:11 -0700

I'm using curl's glob feature to download many short files, say:

curl "http://foo.bar.com/text[1-1000000].txt" > result.txt

I want curl to echo the URL of each file in result.txt. Example:

=> http://foo.bar.com/text17.txt <=

[content of text17.txt]

=> http://foo.bar.com/text18.txt <=

[content of text18.txt]

and so on.

I realize I could use "-o text#1.txt", but I don't want to create a
million files.

I tried "-w %url_effective", but it literally put the string
"%url_effective" in my output file and didn't escape it.

How can I make curl do what I want?

-- 
We're just a Bunch Of Regular Guys, a collective group that's trying
to understand and assimilate technology. We feel that resistance to
new ideas and technology is unwise and ultimately futile.
-------------------------------------------------------------------
List admin: http://cool.haxx.se/cgi-bin/mailman/listinfo/curl-users
FAQ:        http://curl.haxx.se/docs/faq.html
Etiquette:  http://curl.haxx.se/mail/etiquette.html
Received on 2009-05-30