cURL / Mailing Lists / curl-and-php / Single Mail


data scrapping help

From: haroon ahmad <>
Date: Fri, 17 Aug 2007 10:25:43 +0500


I have to browse a site which has SE friendly URLs on the pattern

this kind of URL contains different subcategories names that can be clicked
to form a URL like

then this kind of URL contains different names which produce URLs like

and my main goal is to fetch information from this final URLs

so I want to write a script which can browse the website for all available
'subcat' and 'name' combinations and read data from the name pages i.e*/*.htm

can anybody guide me please how can i proceed for this kind of automatic
process that can browse all urls one by one and then fetch data, i can write
the fetching data part with cURL only problem is the logic for browsing urls
one by one.

thank you

Received on 2007-08-17