cURL / Mailing Lists / curl-library / Single Mail

curl-library

Re: Web crawler in c++

From: Michael Swan <michael_swan_at_optusnet.com.au>
Date: Mon, 3 Oct 2005 22:00:13 +1000

I'm doing some experimental AI, and I need tonnes of data. I'll take a look
at some of your suggestions. If anything else crops up that you think might
help let me know.

----- Original Message -----
From: "Jason Pump" <jpump_at_mindspring.com>
To: "libcurl development" <curl-library_at_cool.haxx.se>
Sent: Monday, October 03, 2005 4:03 AM
Subject: RE: Web crawler in c++

> I have written one, but can't share the source code at this time. If you
> don't have a specific reason I would suggest looking at wget or heratrix.
> Grub is ok, but it is no longer an open source project and the last open
> source release was still quite buggy. The new release is more stable but
not
> os. Depending on what info you need have a look at the amazon, google, and
> grub api's.
>
> > -----Original Message-----
> > From: curl-library-bounces_at_cool.haxx.se
> > [mailto:curl-library-bounces_at_cool.haxx.se]On Behalf Of Alexander Lazic
> > Sent: Sunday, October 02, 2005 3:38 AM
> > To: libcurl development
> > Subject: Re: Web crawler in c++
> >
> >
> > Hi,
> >
> > On Son 02.10.2005 20:26, Michael Swan wrote:
> > >
> > >Has anyone written and got source code to a Web crawler/Spider in C++
> > >and Curl?
> >
> > Do you know about http://grub.org/
> >
> > Hth
> >
> > al ;-)
>
>
>
> --
> No virus found in this incoming message.
> Checked by AVG Anti-Virus.
> Version: 7.0.344 / Virus Database: 267.11.9/116 - Release Date: 30/09/2005
>
>

-- 
No virus found in this outgoing message.
Checked by AVG Anti-Virus.
Version: 7.0.344 / Virus Database: 267.11.9/116 - Release Date: 30/09/2005
Received on 2005-10-03