curl-and-php
Re: post then get help
Date: Tue, 8 Jul 2008 17:21:31 -0400
Yes, I just copied that bit from the function, frankly I don't see what it
does. If anything, I'd be checking for a 200 OK response code.
Maybe just take out the brackets, ya
-Stephen
2008/7/8 ryan pal <ryanpal_at_mail.com>:
> it seems there isn't a 301 or 302. here is the actual flow from firefox
> http live headers (i omitted the header and other junk)
>
> http://<url>/doctypesearch.aspx <http://%3Curl%3E/doctypesearch.aspx>
> POST /doctypesearch.aspx HTTP/1.1
> Host: <url>
> ...some stuff
> Referer: http://<url>/doctypesearch.aspx<http://%3Curl%3E/doctypesearch.aspx>
> Cookie: ASP.NET_SessionId=rdt0fp2ippebau55gwta2y55
> Content-Type: application/x-www-form-urlencodedContent-Length: 164
>
> __VIEWSTATE=dDwyMDY1NDQwMTMxOztsPGNoa1VudmVyaWZpZWQ7Pj5APoilxKEX3nDtx3%2Bj9TB4hNF%2Flw%3D%3D&txtStart=7%2F7%2F2008&txtEnd=7%2F7%2F2008&btnSearchDoc=Search&docTypes=
> HTTP/1.x 200 OK
> ...some stuff
> ----------------------------------------------------------
> http://<url>/results.aspx <http://%3Curl%3E/results.aspx>
> GET /results.aspx HTTP/1.1
> Host: <url>
> ...some stuff
> Referer: http://<url>/doctypesearch.aspx<http://%3Curl%3E/doctypesearch.aspx>
> Cookie: ASP.NET_SessionId=rdt0fp2ippebau55gwta2y55
> HTTP/1.x 200 OK
> ...some stuff
> ----------------------------------------------------------
> http://<url>/css/calibrate_page1_files/sub_topstrip.jpg<http://%3Curl%3E/css/calibrate_page1_files/sub_topstrip.jpg>
> GET /css/calibrate_page1_files/sub_topstrip.jpg HTTP/1.1
> Host: <url>
> ...some stuff
> HTTP/1.x 404 Not Found
> ...some stuff
> ----------------------------------------------------------
> http://<url>/treeView.aspx?initCriteria<http://%3Curl%3E/treeView.aspx?initCriteria>
> =
> GET /treeView.aspx?initCriteria= HTTP/1.1
> Host: <url>
> ...some stuff
> HTTP/1.x 200 OK
> ...some stuff
> ----------------------------------------------------------
> http://<url>/ig_common/20043/scripts/ig_WebGrid_srt.js<http://%3Curl%3E/ig_common/20043/scripts/ig_WebGrid_srt.js>
> GET /ig_common/20043/scripts/ig_WebGrid_srt.js HTTP/1.1
> Host: <url>
> ...some stuff
> HTTP/1.x 200 OK
> ...some stuff
> ----------------------------------------------------------
>
> i don't know how to handle the above. a 200ok is received instead of 301
> or 301 does that mean i should just grab the header? and remove the if
> brackets?
>
> ----- Original Message -----
> From: "Stephen Pynenburg"
> To: "curl with PHP"
> Subject: Re: post then get help
> Date: Tue, 8 Jul 2008 16:25:35 -0400
>
> Ya, my mistake - I left the return variable the same as the input variable.
> In this section:
> if ($http_code == 301 || $http_code == 302)
> {
> echo "code is 301 or 302";
> $matches = array();
> preg_match('/Location:(.*?)\n/', $header, $matches);
> $url = @parse_url(trim(array_pop($matches)));
> }
> return $url;
> Change the variable $url to something else, like $url2, etc.
> Also, it's possible that
> if ($http_code == 301 || $http_code == 302)
> Isn't evaluating properly, so you might just want to take out that whole
> conditional block.
>
> -Stephen
> 2008/7/8 ryan pal <ryanpal_at_mail.com>:
>
>> i see what you are referring to now.
>>
>> when testing your code it returns me the original url that i send the
>> curl_POST_redirect function, not the redirect one.
>>
>>
>>
>>
>>
>> ----- Original Message -----
>> From: "Stephen Pynenburg"
>> To: "curl with PHP"
>> Subject: Re: post then get help
>> Date: Mon, 7 Jul 2008 22:28:29 -0400
>>
>> I was thinking - do the POST request, get the *first* returned
>> redirection URL, then use that URL for standard (GET) requests, using the
>> full curl_redir_exec function:
>>
>> <?PHP
>> //Use this function to POST the page and parse the redirect location
>> header out of the response
>> function curl_POST_grab_redirect($url,$postfields){
>> $ch = curl_init(); //initialize curl handle
>> curl_setopt($ch, CURLOPT_URL, $url); //set url to post to
>> curl_setopt($ch, CURLOPT_VERBOSE, 1);
>> curl_setopt($ch, CURLOPT_FAILONERROR, 1);
>> curl_setopt($ch, CURLOPT_COOKIEFILE, "/tmp");
>> curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1); //return into variable
>> curl_setopt($ch, CURLOPT_TIMEOUT, 24); //timeout after 25s
>> curl_setopt($ch, CURLOPT_POST, 1); //set POST method
>> curl_setopt($ch, CURLOPT_POSTFIELDS, $postfields); //add post fields
>> curl_setopt($ch, CURLOPT_AUTOREFERER, 1);
>> //curl_setopt($ch, CURLINFO_HEADER_OUT, true);
>> $result = curl_exec($ch); //run the whole process
>> list($header, $data) = explode("\n\n", $data, 2);
>> $http_code = curl_getinfo($ch, CURLINFO_HTTP_CODE);
>> //echo "HTTP CODE: $http_code<br>";
>> if ($http_code == 301 || $http_code == 302)
>> {
>> echo "code is 301 or 302";
>> $matches = array();
>> preg_match('/Location:(.*?)\n/', $header, $matches);
>> $url = @parse_url(trim(array_pop($matches)));
>> }
>> return $url;
>> curl_close($ch);
>> unset($ch); //clean up
>> }
>>
>> //POST and get the url
>> $theurl = curl_POST_grab_redirect("http://yoursite.com/page.php
>> ","key=value&so=on&so=forth");
>> //proceed with curl_redir_exec
>> $ch2 = curl_init();
>> curl_setopt($ch2, CURLOPT_GET, 1); //this is set by default anyway
>> curl_setopt($ch2, CURLOPT_URL, $theurl); //from the function
>> /* Add any other options as needed */
>> $res = curl_redir_exec($ch2);
>> ?>
>>
>> 2008/7/7 ryan pal <ryanpal_at_mail.com>:
>>
>>> In order to first post I must tell curl to use POST, which I have done
>>> with:
>>>
>>> curl_setopt(
>>> $ch, CURLOPT_POSTFIELDS, $postfields);
>>>
>>> Since I need to do a GET after this, don't I have to do a CURLPT_HTTPGET
>>> after or else it will still try to post?
>>>
>>> I believe that's the issue I'm having from the beginning. How to do a
>>> POST then a get. I'm under the impression that I have to write code to
>>> switch between the two. I'm also a little confused of I should code what
>>> you describe.
>>>
>>> Here is my main function along with the one you describe. How would I
>>> modify it to do GET?
>>>
>>> list($header, $data) = explode("\n\n", $data, 2);
>>>
>>> $http_code = curl_getinfo($ch, CURLINFO_HTTP_CODE);
>>>
>>> //echo "HTTP CODE: $http_code<br>";
>>>
>>> if ($http_code == 301 || $http_code == 302
>>> )
>>>
>>> {
>>>
>>> curl_setopt(
>>> $ch, CURLOPT_POST, 0); //turn off POST method
>>>
>>> curl_setopt(
>>> $ch, CURLOPT_HTTPGET, 1); //set GET method
>>>
>>> //curl_setopt($ch, CURLOPT_DEBUGFUNCTION,1);
>>>
>>> curl_setopt(
>>> $ch, CURLOPT_POSTFIELDS, NULL);
>>>
>>> echo "code is 301 or 302";
>>>
>>> $matches = array();
>>>
>>> preg_match(
>>> '/Location:(.*?)\n/', $header, $matches);
>>>
>>> $url = @parse_url(trim(array_pop($matches)));
>>>
>>> }
>>>
>>> function
>>> curl_navigate($url,$postfields){
>>>
>>> $ch = curl_init(); //initialize curl handle
>>>
>>> curl_setopt(
>>> $ch, CURLOPT_URL,$url); //set url to post to
>>>
>>> curl_setopt(
>>> $ch, CURLOPT_VERBOSE, 1);
>>>
>>> curl_setopt(
>>> $ch, CURLOPT_FAILONERROR, 1);
>>>
>>> curl_setopt($ch, CURLOPT_COOKIEFILE, "/tmp");
>>>
>>> curl_setopt(
>>> $ch, CURLOPT_RETURNTRANSFER, 1); //return into variable
>>>
>>> curl_setopt(
>>> $ch, CURLOPT_TIMEOUT, 24); //timeout after 25s
>>>
>>> curl_setopt(
>>> $ch, CURLOPT_POST, 1); //set POST method
>>>
>>> curl_setopt(
>>> $ch, CURLOPT_POSTFIELDS, $postfields); //add post fields
>>>
>>> curl_setopt(
>>> $ch, CURLOPT_AUTOREFERER, 1);
>>>
>>> //curl_setopt($ch, CURLINFO_HEADER_OUT, true);
>>>
>>> curl_redir_exec(
>>> $ch);
>>>
>>> $result = curl_exec($ch); //run the whole process
>>>
>>> return $result;
>>>
>>> curl_close(
>>> $ch);
>>>
>>> }
>>>
>>>
>>>
>>>
>>> ----- Original Message -----
>>> From: "Stephen Pynenburg"
>>> To: "curl with PHP"
>>> Subject: Re: post then get help
>>> Date: Mon, 7 Jul 2008 21:43:07 -0400
>>>
>>> Hey,
>>> After the first POST, you only need that little snippet from the function
>>> to get the new Location header.
>>> Once you've got the new URL from your special POST request, you can
>>> actually just use the full function for the rest of your requests.
>>> About using GET, unless you explicitly tell cURL to use POST or PUT or
>>> something, it defaults to a GET request, so you shouldn't have to worry.
>>>
>>> -Stephen
>>>
>>> 2008/7/7 ryan pal <ryanpal_at_mail.com>:
>>>
>>>> stephen,
>>>>
>>>> i see you displayed only a little bit of the redirect function. are you
>>>> suggesting i omit the rest?
>>>>
>>>> also, how do you do a get request? use the CURLOPT_HTTPGET?
>>>>
>>>> oh and one more thing. should this be a seperate function since you
>>>> mention i should do the POST as normal?
>>>>
>>>> thanks,
>>>> ryan
>>>>
>>>> ----- Original Message -----
>>>> From: "Stephen Pynenburg"
>>>> To: "curl with PHP"
>>>> Subject: Re: post then get help
>>>> Date: Mon, 7 Jul 2008 18:39:08 -0400
>>>>
>>>> My recommendation would be to do your first POST request normally, then
>>>> take the data you get and use a little bit of the redir_exec function:
>>>>
>>>> list($header, $data) = explode("\n\n", $data, 2);
>>>> $http_code = curl_getinfo($ch, CURLINFO_HTTP_CODE);
>>>> if ($http_code == 301 || $http_code == 302) {
>>>> $matches = array();
>>>> preg_match('/Location:(.*?)\n/ ', $header, $matches);
>>>> $url = @parse_url(trim(array_pop($matches)));
>>>>
>>>> To get the next URL, then just do a while loop, with a cURL GET request
>>>> in it, until there are no more URLs.
>>>>
>>>> -Stephen
>>>>
>>>>
>>>> 2008/7/7 ryan pal <ryanpal_at_mail.com>:
>>>>
>>>>> i'm trying to figure out how i can do a "post" then a "get". due to
>>>>> restrictions on my server, i have to use the redirect script found on
>>>>> php.net:
>>>>>
>>>>> function curl_redir_exec($ch,$debug="")
>>>>> {
>>>>> static $curl_loops = 0;
>>>>> static $curl_max_loops = 20; if ($curl_loops++ >=
>>>>> $curl_max_loops)
>>>>> {
>>>>> $curl_loops = 0;
>>>>> return FALSE;
>>>>> }
>>>>> curl_setopt($ch, CURLOPT_HEADER, true);
>>>>> curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
>>>>> $data = curl_exec($ch);
>>>>> list($header, $data) = explode("\n\n", $data, 2);
>>>>> $http_code = curl_getinfo($ch, CURLINFO_HTTP_CODE);
>>>>> if ($http_code == 301 || $http_code == 302) {
>>>>> $matches = array();
>>>>> preg_match('/Location:(.*?)\n/', $header, $matches);
>>>>> $url = @parse_url(trim(array_pop($matches)));
>>>>> //print_r($url);
>>>>> if (!$url)
>>>>> {
>>>>> //couldn't process the url to redirect to
>>>>> $curl_loops = 0;
>>>>> return $data;
>>>>> }
>>>>> $last_url = parse_url(curl_getinfo($ch,
>>>>> CURLINFO_EFFECTIVE_URL));
>>>>> /* if (!$url['scheme'])
>>>>> $url['scheme'] = $last_url['scheme'];
>>>>> if (!$url['host'])
>>>>> $url['host'] = $last_url['host'];
>>>>> if (!$url['path'])
>>>>> $url['path'] = $last_url['path'];*/
>>>>> $new_url = $url['scheme'] . '://' . $url['host'] .
>>>>> $url['path'] . ($url['query']?'?'.$url['query']:'');
>>>>> curl_setopt($ch, CURLOPT_URL, $new_url);
>>>>> // debug('Redirecting to', $new_url); return
>>>>> curl_redir_exec($ch);
>>>>> } else {
>>>>> $curl_loops=0;
>>>>> return $data;
>>>>> }
>>>>> i have my curl_setopt to POST, but need to do GET also. how do setup
>>>>> my code so that i post and then do a get for the next few redirects?
>>>>>
>>>>> -- Be Yourself @ mail.com!
>>>>> Choose From 200+ Email Addresses
>>>>> Get a *Free* Account at www.mail.com<http://www.mail.com/Product.aspx>
>>>>> !
>>>>>
>>>>> _______________________________________________
>>>>> http://cool.haxx.se/cgi-bin/mailman/listinfo/curl-and-php
>>>>>
>>>>>
>>>>
>>>> _______________________________________________
>>>> http://cool.haxx.se/cgi-bin/mailman/listinfo/curl-and-php
>>>>
>>>>
>>>> -- Be Yourself @ mail.com!
>>>> Choose From 200+ Email Addresses
>>>> Get a *Free* Account at www.mail.com <http://www.mail.com/Product.aspx>
>>>> !
>>>>
>>>> _______________________________________________
>>>> http://cool.haxx.se/cgi-bin/mailman/listinfo/curl-and-php
>>>>
>>>>
>>>
>>> _______________________________________________
>>> http://cool.haxx.se/cgi-bin/mailman/listinfo/curl-and-php
>>>
>>>
>>> -- Be Yourself @ mail.com!
>>> Choose From 200+ Email Addresses
>>> Get a *Free* Account at www.mail.com <http://www.mail.com/Product.aspx>!
>>>
>>> _______________________________________________
>>> http://cool.haxx.se/cgi-bin/mailman/listinfo/curl-and-php
>>>
>>>
>>
>> _______________________________________________
>> http://cool.haxx.se/cgi-bin/mailman/listinfo/curl-and-php
>>
>>
>> -- Be Yourself @ mail.com!
>> Choose From 200+ Email Addresses
>> Get a *Free* Account at www.mail.com <http://www.mail.com/Product.aspx>!
>>
>> _______________________________________________
>> http://cool.haxx.se/cgi-bin/mailman/listinfo/curl-and-php
>>
>>
>
> _______________________________________________
> http://cool.haxx.se/cgi-bin/mailman/listinfo/curl-and-php
>
>
> -- Be Yourself @ mail.com!
> Choose From 200+ Email Addresses
> Get a *Free* Account at www.mail.com <http://www.mail.com/Product.aspx>!
>
> _______________________________________________
> http://cool.haxx.se/cgi-bin/mailman/listinfo/curl-and-php
>
>
_______________________________________________
http://cool.haxx.se/cgi-bin/mailman/listinfo/curl-and-php
Received on 2008-07-08