cURL / Mailing Lists / curl-users / Single Mail

curl-users

Using curl for spidering

From: ravinder dharmapuram <dharmapu_at_usc.edu>
Date: Sun, 28 Nov 2004 15:06:51 -0800

hi,

We have used Curl as an effective tool to spider web sites.
we can handle cookies and also output header formats as required.
However sites like "expedia" present a problem.
When the user selects a choice for the flight then i think a dll file gets executed.
Also user gets redirected to another page .

Can curl be used to handle such sites ?
Page redirection can be handled using the -L option in other sites.

Any help would be appreciated

Regards
Ravinder

Received on 2004-11-29