cURL / Mailing Lists / curl-users / Single Mail

curl-users

Re: cURL & Monster.com

From: Dan Fandrich <dan_at_coneharvesters.com>
Date: Thu, 14 Jul 2005 11:22:44 -0700

On Thu, Jul 14, 2005 at 10:09:21AM -0700, B Z wrote:
> >Rather than guessing, why not use Live Headers to see the actual request that
> >Firefox sends for the query. That will show you exactly what you will need
> >to emulate using a curl command. Once you've accomplished that, you can then
> >start figuring out whether you'll need to parse Javascript to get that
> >data.
>
> Thanks for the advice. I was unaware of that very useful tool.
>
> So - what your saying is i should have my curl command visit all the specified
> URLS that show up in the Live Headers log?

Only the ones that you're interested in. Most of the URLs downloaded in your
log are for ads and images. If you want those, then yes, you'll need to
download them. Changes are you only want the HTML so find out which one
in your log corresponds to that page and get curl to download it only.

>>> Dan

-- 
http://www.MoveAnnouncer.com              The web change of address service
          Let webmasters know that your web site has moved
Received on 2005-07-14