Recursive file download

From: <>
Date: Sat, 10 Nov 2007 06:38:53 +0000

I know this is two posts in a row, but they are separate questions, and I think that it would make it easier to have them as separate emails/posts. Anyways, on to the question. :)

What's the best way to implement recursive file downloading in curl? I didn't check whether pycurl does it (licensing is still an issue, so that option is out).

Do I need to roll my own solution?
For example, do a search for the <a tags.... and extract the urls or
find a Python sgml/html parser and extract the urls it finds

or are there several more robust solutions already available in Python?
Received on 2007-11-10