cURL / Mailing Lists / curl-and-php / Single Mail

curl-and-php

cURL opt is not correct or missing?

From: David Colter <dolan2go_at_yahoo.com>
Date: Tue, 27 Nov 2007 21:18:12 -0800 (PST)

I'm really new to cURL, but have been reading everywhere I can for the solution to my problem. So far the understanding evades me.

I am trying to get data from my companies website that requires a user id and password. I think I've written the code to get beyond the entry page and download the resulting page, but that is not being returned. So, I think that I'm missing something in the pages Method or the ASPSESSION is not being handled by cURL. Does this occur when the form is not correctly sent/received? Incidentally, no cookies are being written. I think there's a problem with 'targetURL' being part of $form_data. Another part of the confusion results from what I can see being sent by using HTTP Headers (Firefox extension) The first bit of headers I captured:

https://www.aapilots.com/public/flash/login.asp

GET /public/flash/login.asp HTTP/1.1
Host: www.aapilots.com
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.0.12) Gecko/20070508 Firefox/1.5.0.12
Accept: text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5
Accept-Language: en-us,en;q=0.5
Accept-Encoding: gzip,deflate
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Keep-Alive: 300
Connection: keep-alive
Referer: https://www.aapilots.com/public/flash/home.html
Cookie: ASPSESSIONIDACBTARDD=CGGIIMJDPKNOMPFJIFIPPKLP; AXCOOKIELOGIN=%3C354611%3EXBTV%2EEgEyK9JEhYJ3ZX98%2F
Cache-Control: max-age=0

HTTP/1.x 200 OK
Date: Sat, 24 Nov 2007 03:37:47 GMT
Server: Microsoft-IIS/6.0
X-Powered-By: ASP.NET
Content-Length: 7097
Content-Type: text/html
Cache-Control: private

************ Beginning On Index

https://www.aapilots.com/ps_login.asp

POST /ps_login.asp HTTP/1.1
Host: www.aapilots.com
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.0.12) Gecko/20070508 Firefox/1.5.0.12
Accept: text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5
Accept-Language: en-us,en;q=0.5
Accept-Encoding: gzip,deflate
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Keep-Alive: 300
Connection: keep-alive
Referer: https://www.aapilots.com/public/flash/login.asp
Cookie: ASPSESSIONIDACBTARDD=CGGIIMJDPKNOMPFJIFIPPKLP; AXCOOKIELOGIN=%3C354611%3EXBTV%2EEgEyK9JEhYJ3ZX98%2F
Content-Type: application/x-www-form-urlencoded
Content-Length: 62
targetURL=&frmLoginUser=354611&frmLoginPass=not the real one&x=34&y=12

HTTP/1.x 302 Object moved
Date: Sat, 24 Nov 2007 03:39:46 GMT
Server: Microsoft-IIS/6.0
X-Powered-By: ASP.NET
Location: private/index.asp
Content-Length: 138
Content-Type: text/html
Set-Cookie: AXCOOKIELOGIN=%3C354611%3EXBTV%2EEgEyK9JEhYJ3ZX98%2F; path=/
Cache-Control: private
https://www.aapilots.com/private/index.asp

GET /private/index.asp HTTP/1.1
Host: www.aapilots.com
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.0.12) Gecko/20070508 Firefox/1.5.0.12
Accept: text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5
Accept-Language: en-us,en;q=0.5
Accept-Encoding: gzip,deflate
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Keep-Alive: 300
Connection: keep-alive
Referer: https://www.aapilots.com/public/flash/login.asp
Cookie: ASPSESSIONIDACBTARDD=CGGIIMJDPKNOMPFJIFIPPKLP; AXCOOKIELOGIN=%3C354611%3EXBTV%2EEgEyK9JEhYJ3ZX98%2F

HTTP/1.x 200 OK
Date: Sat, 24 Nov 2007 03:39:46 GMT
Server: Microsoft-IIS/6.0
X-Powered-By: ASP.NET
Cache-Control: private, private, private, private, private, no-cache
Expires: 0, 0, 0, 0, Fri, 23 Nov 2007 03:39:46 GMT
Pragma: no-cache
Content-Length: 748
Content-Type: text/html
https://www.aapilots.com/private/index.asp

****************

https://www.aapilots.com/private/index.asp

***************

https://www.aapilots.com/private/nav.asp

GET /private/nav.asp HTTP/1.1
Host: www.aapilots.com
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.0.12) Gecko/20070508 Firefox/1.5.0.12
Accept: text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5
Accept-Language: en-us,en;q=0.5
Accept-Encoding: gzip,deflate
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Keep-Alive: 300
Connection: keep-alive
Referer: https://www.aapilots.com/private/index.asp
Cookie: ASPSESSIONIDACBTARDD=CGGIIMJDPKNOMPFJIFIPPKLP; AXCOOKIELOGIN=%3C354611%3EXBTV%2EEgEyK9JEhYJ3ZX98%2F

HTTP/1.x 200 OK
Date: Sat, 24 Nov 2007 03:39:46 GMT
Server: Microsoft-IIS/6.0
X-Powered-By: ASP.NET
Cache-Control: private, private, private, private, private
Expires: 0, 0, 0, 0
Content-Length: 5788
Content-Type: text/html

The script is at flydata.mobi/webbot.php and the output will show much of the details. Here is the code of webbot.php (some omitted):

# Select cookie file (based on username)
$cookie_file = "/home/flydat5/public_html/cookies/".$user."cookies.txt";

# Go Stealthy
$WEBBOT_NAME = "Internet Explorer";

# Request the Login Page
$target = "https://www.aapilots.com/public/flash/login.asp";
$ref = "https://aapilots.com/public/flash/home.html";
# $page_array = http_get($target, $ref);

# Begin Sending Login Info (assuming Cookie Authentication p. 204)

# Define the login form data
$form_data = "targetURL=&enter=Login&frmLoginUser=354611&frmLoginPass=not the real one&x=0&y=0";

# Create cURL Session
$ch = curl_init();

if (!$ch) {
die("Couldn't initialize a cURL handle");
} else {
echo 'curl handle initialized ';
}
echo "target = " . $target . " cookie file = " . $cookie_file . " webbot name = " . $WEBBOT_NAME . " ";
curl_setopt($ch, CURLOPT_USERAGENT, $WEBBOT_NAME); // See above Go Stealthy
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, FALSE); // Needed for https: & no certificate
curl_setopt($ch, CURLOPT_URL, $target); // Define target site
curl_setopt($ch, CURLORT_REFERER, $ref); // Define refering page
curl_setopt($ch, CURLOPT_RETURNTRANSFER, TRUE); // Return page in String
curl_setopt($ch, CURLOPT_COOKIEJAR, $cookie_file); // Where to WRITE cookies
curl_setopt($ch, CURLOPT_COOKIEFILE, $cookie_file); // Where to READ cookies FROM
curl_setopt($ch, CURLOPT_HEADER, TRUE);
curl_setopt($ch, CURLOPT_POST, TRUE); // **** Must confirm POST method used ****
curl_setopt($ch, CURLOPT_POSTFIELDS, $form_data);
curl_setopt($ch, CURLOPT_HEADER, TRUE);
curl_setopt($ch, CURLOPT_NOBODY, TRUE);
curl_setopt($ch, CURLOPT_UNRESTRICEDAUTH, TRUE);
# curl_setopt($ch, CURLOPT_UPLOAD, TRUE);
# curl_setopt($ch, CURLOPT_VERBOSE, 1);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, TRUE);

# Execute the PHP/CURL session and echo the downloaded page, info and errors
$page = curl_exec($ch);

echo $page . " ";
echo "<xmp>" . $page . "</xmp> "; // Output the page with HTML tags
$errors = curl_error($ch);

if (!$errors) {
 echo "There are NO ERRORS ";
 }
else {
 echo "The cURL Errors using print( ) is <pre>";
 print_r($errors);
 echo "</pre> ";
}
$info_array = curl_getinfo($ch);
echo " The cURL info using print( ) is <pre>";
print_r($info_array);
echo "</pre> ";

# At this point webbot should be at AAPilots.com Home Page. Ready to jump into Full Screen Sabre
# Request Sabre using the Session Variable
# $target2 = https://www.aapilots.com/private/sabrenew.asp";
# $page_array = http_get($target2, $ref="");
# echo " The $return array [FILE] = " . $return_array['FILE']." ";
# echo " $return array [STATUS] = " . $return_array['STATUS']. " ";

# Close the cURL session
curl_close($ch);

Sorry for the noobie plea for help. Thanks for reading this.
David

____________________________________________________________________________________
Never miss a thing. Make Yahoo your home page.
http://www.yahoo.com/r/hs

_______________________________________________
http://cool.haxx.se/cgi-bin/mailman/listinfo/curl-and-php
Received on 2007-11-28

This message: [ Message body ]
Next message: Daniel Stenberg: "Re: cURL opt is not correct or missing?"
Previous message: Daniel Stenberg: "Re: Is there a trace or trace-ascii option for PHP cURL?"
Next in thread: Daniel Stenberg: "Re: cURL opt is not correct or missing?"
Reply: Daniel Stenberg: "Re: cURL opt is not correct or missing?"

Contemporary messages sorted: [ by date ] [ by thread ] [ by subject ] [ by author ] [ by messages with attachments ]