Libcurl would allow you to make HTTP(S) requests. However in order to
build a crawler you would also need to parse the html to extract the
outgoing links from that web page. You would need some html parsing
library to do that. However if you just have the list of URLs to fetch,
that shud be straightforward.

Thanks

Dinesh

________________________________

From: curl-library-bounces_at_cool.haxx.se
[mailto:curl-library-bounces_at_cool.haxx.se] On Behalf Of Kulbhushan
Sharma
Sent: Friday, January 27, 2006 11:22 AM
To: curl-library_at_cool.haxx.se
Subject: Libcurl Help Required

Hi Everyone,

I am a New Joiny to the world of Libcurl Users.

Can anyone please tell me that weather I can build a "Webcrawler" Using
Libcurl. What all I needed.

Please Reply Soon.

Kulbhushan
Received on 2006-01-27