I am trying to spider a site that requires me to accept a cookie. I can
spider it fine with out going thru a proxy but as soon as I go thru the
proxy the site just says please accept cookies.

Code and verbose output below:

$ch = curl_init();
curl_setopt ($ch, CURLOPT_PROXY, $proxy);
curl_setopt($ch,CURLOPT_VERBOSE,true);
curl_setopt($ch,CURLOPT_COOKIEFILE,1);
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS, $first_post);
curl_setopt($ch, CURLOPT_URL,"http://$url");

--------------------------------------------------------
Host: www.host.com
Pragma: no-cache
Accept: */*
Content-Length: 141
Content-Type: application/x-www-form-urlencoded

&lop=0&hip=10000000&bed=0&cou=74&cou=74&slo=undefined&fid=0&bid=1&dbt=1&nre=undefined&thu=undefined&vto=undefined&ord=undefined&nba=undefined< HTTP/1.0 200 OK
< Server: Microsoft-IIS/5.0
< Date: Fri, 11 Feb 2005 13:44:59 GMT
< P3P: CP="CAO DSP COR CUR ADMi CONi TELi OUR LEG UNI NAV INT PRE OTC",policyref="http://www.host.com/w3c/p3p.xml"
< X-Powered-By: ASP.NET
< Pragma: no-cache
< Cache-Control: private
< Content-Length: 13488
< Content-Type: text/html
* Added cookie ASPSESSIONIDSQBARARD="ELJNHBBCNOIFEJKHIPBKLBFO" for domain www.host.com, path /, expire 0
< Set-Cookie: ASPSESSIONIDSQBARARD=ELJNHBBCNOIFEJKHIPBKLBFO; path=/
< Cache-Control: private
< X-Cache: MISS from webmaster.valadou.com
* HTTP/1.0 proxy connection set to keep alive!
< Proxy-Connection: keep-alive
* Connection #0 to host 81.80.43.105 left intact
* Re-using existing connection! (#0) with host 81.80.43.105
* Connected to 81.80.43.105 (81.80.43.105) port 8080
> POST http://www.host.com/home/search/results.asp HTTP/1.1
Host: www.host.com
Pragma: no-cache
Accept: */*
Content-Length: 176
Content-Type: application/x-www-form-urlencoded

&REFINED=1&&lop=0&slo=undefined&hip=20000000&thu=undefined&vto=undefined&bed=0&cou=74, 74&nre=undefined&fid=0&dbt=1&bid=1&ord=undefined&nba=undefined&sav=1&asv=undefined&ref=24< HTTP/1.0 200 OK
< Server: Microsoft-IIS/5.0
< Date: Fri, 11 Feb 2005 13:45:06 GMT
< P3P: CP="CAO DSP COR CUR ADMi CONi TELi OUR LEG UNI NAV INT PRE OTC",policyref="http://www.hosta.com/w3c/p3p.xml"
< X-Powered-By: ASP.NET
< Pragma: no-cache
< Cache-Control: private
< Content-Length: 627
< Content-Type: text/html
* Replaced cookie ASPSESSIONIDSQBARARD="JMJNHBBCNMNOIDBODPLCAJEO" for domain www.host.com, path /, expire 0
< Set-Cookie: ASPSESSIONIDSQBARARD=JMJNHBBCNMNOIDBODPLCAJEO; path=/
< Cache-Control: private
< X-Cache: MISS from webmaster.valadou.com
* HTTP/1.0 proxy connection set to keep alive!
< Proxy-Connection: keep-alive
* Connection #0 to host 81.80.43.105 left intact
* Re-using existing connection! (#0) with host 81.80.43.105
* Connected to 81.80.43.105 (81.80.43.105) port 8080
> POST http://www.host.com/home/search/results.asp HTTP/1.1
Host: www.host.com
Pragma: no-cache
Accept: */*
Content-Length: 177
Content-Type: application/x-www-form-urlencoded
Received on 2005-02-11