cURL / Mailing Lists / curl-users / Single Mail


Re: Smart strategy to write million of files in OSX with bash curl?

From: Rodrigo Zanatta Silva <>
Date: Fri, 16 Jan 2015 22:04:44 -0200

>I would approach the problem by downloading them as individual files using
>-O option (as you presumably are doing currently) with the additional step
>having curl print the file names using the -w option to stdout. I'd then
>stdout to a separate process (like zip or another shell script) which
would use
>the list of file names to consolidate the individual files into a single
large file.

Yeah... I understand what you say and the -w option. I will try using the
database approach. Zip is one solution too... but... I think the database
is a best option.

List admin:
Received on 2015-01-17