Hey David, thanks for getting back,
I've found your suggestion as a solution to posting asp/aspx pages, as I understand asp/aspx http header for cookie is not "set-cookie:" but "cookie:". If I try and save a cookie from asp/aspx website I get nothing.
AS I UNDERSTAND for curl to save cookie http response must come with "set-cookie:" header. Is there any way of getting curl to recognize "cookie:" header?

I tried following php script:
<?
/*
* Author: Ojas Ojasvi
* Released: September 25, 2007
* Description: An example of the disguise_curl() function in order to grab contents from a website while remaining fully camouflaged by using a fake user ag
*/
$url = 'www.bookryanair.com/SkySales/FRSearchRedirect.aspx';
// disguises the curl using fake headers and a fake user agent.
function disguise_curl($url)
{
  $curl = curl_init();
  $cookie_session_id = "WT_FPC=id=2acba6cdf3fab647b521204106034801:lv=1204106303613:ss=1204106034801";
  // Setup headers - I used the same headers from Firefox version 2.0.0.6
  // below was split up because php.net said the line was too long. :/
  $header[0] = "Accept: text/xml,application/xml,application/xhtml+xml,";
  $header[0] .= "text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5";
  $header[] = "Cache-Control: max-age=0";
  $header[] = "Connection: keep-alive";
  $header[] = "Keep-Alive: 300";
  $header[] = "Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7";
  $header[] = "Accept-Language: en-us,en;q=0.5";
  $header[] = "Cookie: $cookie_session_id";
  $header[] = "Pragma: "; // browsers keep this blank.
  curl_setopt($curl, CURLOPT_URL, $url);
  curl_setopt($curl, CURLOPT_POST, 1);
  curl_setopt($curl, CURLOPT_POSTFIELDS, 'travel_type=on&sector1_o=aSTN&sector1_d=KUN&sector_1_d=27&sector_1_m=042008&sector_2_d=00&sector_2_m=--&ADULT=1&CHILD=0&INFANT=0&mode=0&pT=1ADULT&oP=&rP=&nom=1&date1=20080427&date2=&m1=20080427aSTNKUN&m2=&m1DP=0&m1DO=0&m2DP=0&m2DO=0&pM=0&tc=1&module=SB&page=SELECT');
  curl_setopt($curl, CURLOPT_VERBOSE, true);
  curl_setopt($curl, CURLOPT_RETURNTRANSFER, true);
  curl_setopt($curl, CURLOPT_USERAGENT, 'Googlebot/2.1 (+http://www.google.com/bot.html)');
  curl_setopt($curl, CURLOPT_HTTPHEADER, $header);
  curl_setopt($curl, CURLOPT_REFERER, 'http://www.google.com');
  curl_setopt($curl, CURLOPT_ENCODING, 'gzip,deflate');
  curl_setopt($curl, CURLOPT_AUTOREFERER, true);
  curl_setopt($curl, CURLOPT_RETURNTRANSFER, 1);
  curl_setopt($curl, CURLOPT_TIMEOUT, 10);
  $html = curl_exec($curl); // execute the curl command
  curl_close($curl); // close the connection
  return $html; // and finally, return $html
}
// uses the function and displays the text off the website
$text = disguise_curl($url);
echo $text;
?>

where $cookie_session_id is a cookie I'm using that I've captured today using wireshard, strange but there's no ASP.NET_SessionId in it...

When I run the script I'm getting following:

* About to connect() to www.bookryanair.com port 80
* Trying 62.134.190.241... * connected
* Connected to www.bookryanair.com (62.134.190.241) port 80
> POST /SkySales/FRSearchRedirect.aspx HTTP/1.1
User-Agent: Googlebot/2.1 (+http://www.google.com/bot.html)
Host: www.bookryanair.com
Accept-Encoding: gzip,deflate
Referer: http://www.google.com
Accept: text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5
Cache-Control: max-age=0
Connection: keep-alive
Keep-Alive: 300
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Accept-Language: en-us,en;q=0.5
Cookie: WT_FPC=id=2acba6cdf3fab647b521204106034801:lv=1204106303613:ss=1204106034801
Content-Length: 264
Content-Type: application/x-www-form-urlencoded
travel_type=on&sector1_o=aSTN&sector1_d=KUN&sector_1_d=27&sector_1_m=042008&sector_2_d=00&sector_2_m=--&ADULT=1&CHILD=0&INFANT=0&mode=0&pT=1ADULT&oP=&rP=&nom=1&date1=20080427&date2=&m1=20080427aSTNKUN&m2=&m1DP=0&m1DO=0&m2DP=0&m2DO=0&pM=0&tc=1&module=SB&page=SELECT< HTTP/1.1 302 Found
< Connection: close
< Date: Wed, 27 Feb 2008 10:20:51 GMT
< Server: Microsoft-IIS/6.0
< X-Powered-By: ASP.NET
< X-AspNet-Version: 2.0.50727
< Pragma: no-cache
< Pragma: no-cache
< Location: http://www.bookryanair.com/SkySales/FRSelect.aspx
< Set-Cookie: ASP.NET_SessionId=regfijpmxk452juzpwa5zbyn2bfw; path=/; HttpOnly
< Cache-Control: no-cache, no-store
< Pragma: no-cache
< Expires: -1
< Content-Type: text/html; charset=utf-8
< Content-Length: 166
* Closing connection #0
<html><head><title>Object moved</title></head><body>
<h2>Object moved to here.</h2>
</body></html>

Any suggestions are appreciated, thanks very much!!!

----- Original Message ----
From: David Colter <dolan2go_at_yahoo.com>
To: curl with PHP <curl-and-php_at_cool.haxx.se>
Sent: Wednesday, February 27, 2008 2:37:40 AM
Subject: Re: help with curl POST with cookie

Here's what used since I couldn't get curl to manage cookies for me ( I don't remember the reason it wouldn't work).

Parse the returned header to get the cookie value and then add it to the header array similar to this:
$header_array[] = "Cookie: $cookie_session_id";
 which you send with this:
 curl_setopt($ch1, CURLOPT_HTTPHEADER, $header_array); // Send Cookie value obtained previously
Hope that helps.

David
On Feb 26, 2008, at 7:33 AM, Tomas wrote:

If I set CURLOPT_VERBOSE here's what I get on the output...
 

* About to connect() to www.bookryanair.com port 80includes/curl_ryanair_dev.php
* Trying 62.134.190.241... * connected
* Connected to www.bookryanair.com (62.134.190.241) port 80
> POST /SkySales/FRSearchRedirect.aspx HTTP/1.1
Host: www.bookryanair.com
Accept: */*
Content-Length: 264
Content-Type: application/x-www-form-urlencoded
travel_type=on&sector1_o=aSTN&sector1_d=KUN&sector_1_d=27&sector_1_m=042008&sector_2_d=00&sector_2_m=--&ADULT=1&CHILD=0&INFANT=0&mode=0&pT=1ADULT&oP=&rP=&nom=1&date1=20080427&date2=&m1=20080427aSTNKUN&m2=&m1DP=0&m1DO=0&m2DP=0&m2DO=0&pM=0&tc=1&module=SB&page=SELECT< HTTP/1.1 302 Found
< nnCoection: close
< Date: Tue, 26 Feb 2008 12:29:25 GMT
< Server: Microsoft-IIS/6.0
< X-Powered-By: ASP.NET
< X-AspNet-Version: 2.0.50727
< Pragma: no-cache
< Pragma: no-cache
< Location: http://www.bookryanair.com/SkySales/FRSelect.aspx
< Set-Cookie: ASP.NET_SessionId=regfbtd4z045on1gjcjsvbfo4gmo; path=/; HttpOnly
< Cache-Control: no-cache, no-store
< Pragma: no-cache
< Expires: -1
< Content-Type: text/html; charset=utf-8
< Content-Length: 166
* Connection #0 to host www.bookryanair.com left intact
* Closing connection #0
<html><head><title>Object moved</title></head><body>
<h2>Object moved to here.</h2>
</body></html>

----- Original Message ----
From: Tomas <mail2tomas_at_yahoo.com>
To: curl-and-php_at_cool.haxx.se
Sent: Tuesday, February 26, 2008 12:06:41 PM
Subject: help with curl POST with cookie

Hi,
I need some ideas from you please!
The website I used to check cheap flights have changed their booking system from cgi based to aspx, and with that it is not just a simple POST query, it appears I need to have a "cookie".
 
Data caputred with wireshark while trying to book a flight:
Hypertext Transfer Protocol
 POST /SkySales/FRSearchRedirect.aspx?culture=EN-GB HTTP/1.1
  Request Method: POST
  Request URI: /SkySales/FRSearchRedirect.aspx?culture=EN-GB
  Request Version: HTTP/1.1
 Accept: image/gif, image/x-xbitmap, image/jpeg, image/pjpeg, application/x-shockwave-flash, application/vnd.ms-excel, application/vnd.ms-powerpoint, application/msword, */*
 Referer: http://www.ryanair.com/site/EN/
 Accept-Language: en-gb
 Content-Type: application/x-www-form-urlencoded
 Accept-Encoding: gzip, deflate
 User-Agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 1.1.4322; .NET CLR 2.0.50727)
 Host: www.bookryanair.com
 Content-Length: 264
 Connection: Keep-Alive
 Cache-Control: no-cache
 Cookie: WT_FPC=id=62.50.231.14-2356471872.29911443:lv=1204018317334:ss=1204018194423; ASP.NET_SessionId=regfu04fe332nuihir45bc2tyi45
Line-based text data: application/x-www-form-urlencoded
 travel_type=on&sector1_o=aSTN&sector1_d=KUN&sector_1_d=27&sector_1_m=042008&sector_2_d=00&sector_2_m=--&ADULT=1&CHILD=0&INFANT=0&mode=0&pT=1ADULT&oP=&rP=&nom=1&date1=20080427&date2=&m1=20080427aSTNKUN&m2=&m1DP=0&m1DO=0&m2DP=0&m2DO=0&pM=0&tc=1&module=SB&page=SELECT
 
I tried simply POST Line-based text data: application/x-www-form-urlencoded content towww.bookryanair.com/SkySales/FRSearchRedirect.aspx
function curl_ryanair()
{
$ch = curl_init();
/* curl set options */
curl_setopt($ch, CURLOPT_URL, "www.bookryanair.com/SkySales/FRSearchRedirect.aspx");
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS, "travel_type=on&sector1_o=aSTN&sector1_d=KUN&sector_1_d=27&sector_1_m=042008&sector_2_d=00&sector_2_m=--&ADULT=1&CHILD=0&INFANT=0&mode=0&pT=1ADULT&oP=&rP=&nom=1&date1=20080427&date2=&m1=20080427aSTNKUN&m2=&m1DP=0&m1DO=0&m2DP=0&m2DO=0&pM=0&tc=1&module=SB&page=SELECT"
 
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
/* exec curl */
$data = curl_exec ($ch);
/* end curl session */
curl_close($ch);
return $data;
}

but I'm getting a message saying
Object moved to here.
How do I need to fix the script so when I do POST I'm actually posting with that cookie?

Looking for last minute shopping deals? Find them fast with Yahoo! Search.

Looking for last minute shopping deals? Find them fast with Yahoo! Search._______________________________________________
http://cool.haxx.se/cgi-bin/mailman/listinfo/curl-and-php

      ____________________________________________________________________________________
Looking for last minute shopping deals?
Find them fast with Yahoo! Search. http://tools.search.yahoo.com/newsearch/category.php?category=shopping

_______________________________________________
http://cool.haxx.se/cgi-bin/mailman/listinfo/curl-and-php
Received on 2008-02-27