cURL / Mailing Lists / curl-users / Single Mail

curl-users

Re: test curl commands that are/is screwing up..

From: Dan Fandrich <dan_at_coneharvesters.com>
Date: Sun, 31 May 2015 23:31:39 +0200

On Sun, May 31, 2015 at 05:10:22PM -0400, bruce wrote:
> Hi Group.
>
> I've got a short test that "should" work. The test is doing a
> curl/fetch of a page from a site that's running jscript on the back.
>
> The test has stepped through the initial pages/using
> Firefox/LiveHttpheaders to see what the network traffic is actually
> doing. This is replicated in the test curl functions. The app uses
> cookies/SSL, but no user/password login process..
>
> It's actually pretty straightforward as far as I can see.
>
> However, the test is not able to generate the target page, and in fact
> seems to be running into a 302 somewhere.
>
> The goal is is to generate the "page" after the initial pages that has
> the list of the dept alphnumeric selection list.
>
> If someone can point out what I've screwed up, much obliged.
>
> #!/bin/sh -v
> #
> # test shell for wget
> #
>
>
> #test umich
> echo "" > ole.lwp
>
> curl -vvv -A "Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.0.11)
> Gecko/2009061118 Fedora/3.0.11-1.fc9 Firefox/3.0.11" --cookie-jar
> ole.lwp --cookie ole.lwp -L
> "https://csprod.dsc.umich.edu/services/schedofclasses?strm=2060"
>
> curl -vvv -A "Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.0.11)
> Gecko/2009061118 Fedora/3.0.11-1.fc9 Firefox/3.0.11" --cookie-jar
> ole.lwp --cookie ole.lwp -L
> "https://csprod.dsc.umich.edu/psp/csprodpa/EMPLOYEE/HRMS/c/COMMUNITY_ACCESS.M_SR_SC_CLS_SRCH.GBL"
>
>
> curl -vvv -A "Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.0.11)
> Gecko/2009061118 Fedora/3.0.11-1.fc9 Firefox/3.0.11" --cookie-jar
> ole.lwp --cookie ole.lwp -L
> "https://csprod.dsc.umich.edu/psc/csprodpa/EMPLOYEE/HRMS/c/COMMUNITY_ACCESS.CLASS_SEARCH.GBL"
>
> #
> # at this point, the test gets the actual page with the "correct data
> # --the idea is to then get the page that would list
> # the "depts...
> #
> #
> #exit
>
> #
> # this curl should get the page that has the list of the depts..
> # --- THIS is not working as expected...
> #
>
> curl -vvv -A "Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.0.11)
> Gecko/2009061118 Fedora/3.0.11-1.fc9 Firefox/3.0.11" --cookie-jar
> ole.lwp --cookie ole.lwp -e
> "https://csprod.dsc.umich.edu/psc/csprodpa/EMPLOYEE/HRMS/c/COMMUNITY_ACCESS.CLASS_SEARCH.GBL"
> -d "ICAJAX=1&ICNAVTYPEDROPDOWN=0&ICType=Panel&ICElementNum=0&ICStateNum=3&ICAction=CLASS_SRCH_WRK2_SSR_PB_SUBJ_SRCH%240&ICXPos=0&ICYPos=182&ResponsetoDiffFrame=-1&TargetFrameName=None&FacetPath=None&ICFocus=&ICSaveWarningFilter=0&ICChanged=-1&ICResubmit=0&ICSID=JAQZNpudU6JUmDHUTyctshyzD2bx%2Ba6C2lE%2Bmljpf1U%3D&ICActionPrompt=false&ICFind=&ICAddCount=&ICAPPCLSDATA="
> -L "https://csprod.dsc.umich.edu/psc/csprodpa/EMPLOYEE/HRMS/c/COMMUNITY_ACCESS.CLASS_SEARCH.GBL"
>
>
> exit
>
> The cookie/post data is generated from the livehttpheader results..

There's a pretty good chance that there's a mismatch between the
form parameters (possibly generated by Javascript on the page; ICSID is a
suspicious-looking candidate) with one of the cookies. Try manually adding the
cookies you received using Firefox to the cookie jar used by curl as well as
the form parameters (on the same request) and see if that works. If so, then
you'll have to figure out how to automate synchronizing the form parameters
with the cookies in order to script it.

>>> Dan
-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-users
FAQ: http://curl.haxx.se/docs/faq.html
Etiquette: http://curl.haxx.se/mail/etiquette.html
Received on 2015-06-01