cURL / Mailing Lists / curl-users / Single Mail

curl-users

Re: Using cURL to Retrieve BugIDs from Cisco's Web Site

From: <denis.papathanasiou_at_gmail.com>
Date: Mon, 20 Oct 2008 19:01:21 -0400

Josh Anderson wrote:
> I am trying to retrieve BugID pages from Cisco's Web Site. For some
> reason, I am having trouble logging into the site to retrieve a link
> such as:
>
> http://tools.cisco.com/Support/BugToolKit/search/getBugDetails.do?method=fetchBugDetails&bugId=CSCsi78581
>
> Here is the current iteration I am using (I've tried numerious variations):
>
> curl -c cookies.txt -b cookies.txt -L -k -d
> "USER=usernamehere&PASSWORD=passwordhere&target=http://cisco.com/cgi-bin/login&smauthreason=0&smquerydata=&smagentname=zjGKGqr62shoVBG6cNUdYNajdKPzmOFLa/ZkeebT0+NV+EcoXFhv/vB8k65Cw/+x&postpreservationdata=&SMENC=ISO-8859-1&SMLOCALE=US-EN"
> "http://cisco.com/cgi-bin/login"
> "http://tools.cisco.com/Support/BugToolKit/search/getBugDetails.do?method=fetchBugDetails&bugId=CSCsi78581"
>
> All I get is the login page two times. I'm sure I am way off here, but
> if anyone has any suggestions I would greatly appreciate it. Thanks!

It looks like Cisco's login form has a bunch of hidden variables which
are not being sent in your curl request:

> <input name="nv" value="Search All cisco.com##cisco.com" type="hidden" />
> <input name="language" value="en" type="hidden" />
> <input name="country" value="US" type="hidden" />
> <input name="accessLevel" value="Guest" type="hidden" />

Also, some servers won't reply to a form request unless they see a user
agent string (http://en.wikipedia.org/wiki/User_agent), so you may need
to provide that with curl's -A parameter
(http://curl.haxx.se/docs/manpage.html#-A--user-agent).
-------------------------------------------------------------------
List admin: http://cool.haxx.se/cgi-bin/mailman/listinfo/curl-users
FAQ: http://curl.haxx.se/docs/faq.html
Etiquette: http://curl.haxx.se/mail/etiquette.html
Received on 2008-10-21