On Sun, May 25, 2008 at 9:37 PM, Georg Lippold <
georg.lippold_at_student.qut.edu.au> wrote:

> Hello,
>
> I'm experiencing a strange problem when using curl to submit some data
> to a publicly accessible https secured web page. The web page uses a
> session cookie and a base-64 encoded blob to pre-authenticate the user.
> Once the blob and the cookie are on the client and the blob has been
> posted back to the web page, a login form is presented.
>
> When I access the page using a web browser like lynx or w3m, I have no
> problem to get to the login form. However, if I try to use curl to
> access the web page, I always get the equivalent to a HTTP-500. There
> are a few redirects being used by the page, so the URL changes in the
> process, but I figure that curl can handle this. However, the first
> server that I have to connect to sends a "Connection: Keep-Alive" header
> and I don't know if the process fails because of this. I do not see an
> option in curl how to handle this.
>
> The original web page is at http://access.qut.edu.au and my curl script
> to access the site is (in bash syntax):
>
> #############################################################
>
> #!/bin/sh
> COOKIES="`mktemp auth-cookies-XXXXXXX`"
> outfile1=access-1.html
> SESSION_CONTROLLER="https://esoe.qut.edu.au/sso"
> TARGET_URL="http://access.qut.edu.au"
> curl -j -L -k -i -c ${COOKIES} --compressed "${TARGET_URL}" > $outfile1
> SESSION_ID=`grep -a SAMLRequest ${outfile1} | grep value | awk
> -F"value=" '{print $2}' | sed s,"/>$",,g | sed s,"^\"",,g | sed s,"\"$",,g`
> # leading/trailing "
> REFERER="`grep -a Location: ${outfile1} | awk '{print $NF}'`"
> echo "------------------ SESSION ID ---------------------"
> echo ${SESSION_ID}
> echo "------------------ SESSION ID ---------------------"
> echo "------------------ REFERER ---------------------"
> echo ${REFERER}
> echo "------------------ REFERER ---------------------"
> echo "------------------ COOKIES ---------------------"
> cat $COOKIES
> echo "------------------ COOKIES ---------------------"
> curl --trace - -k -i -b ${COOKIES} -c ${COOKIES} \
> -L \
> -e "${REFERER}" \
> -A "Mozilla/4.0" \
> --data "SAMLRequest=${SESSION_ID}" \
> --data "submit=Verify Identity" \
> -m 90 \
> --digest \
> -D header.html \
> ${SESSION_CONTROLLER}
>
> #############################################################
>
> Does anybody have any idea why the connection does not work with curl
> while it works with any browser I tried?
>

Have you tried using LiveHTTPheaders in Firefox to see exactly what it sends
to the server??

There's some javascript on the page that fools around with cookies and
stuff. LiveHTTPheaders will will show you the end result of that.

For example, the loader function deletes the spepAutoSubmit cookie.

You should also url-encode the data for the post:

   -d "submit=Verify+Identity"

I don't think that fixes it, but you should do it anyway

Ralph Mitchell
Received on 2008-05-26