curl-and-php
curl not scraping Microsoft.com (something wrong?)
Date: Wed, 13 Apr 2005 12:35:54 -0400 (EDT)
Using the code below, I get an error page rather than the search page that
I'd expect. Cookies, referrer pages, etc don't seem to make a difference.
Could anyone help me figure out why it's not working?
Thank you
-Daniel-
<?php
$ch = curl_init();
curl_setopt ($ch, CURLOPT_URL,
"http://directory.microsoft.com/mprd/Search.aspx?action=search");
curl_setopt ($ch, CURLOPT_USERAGENT,
"Mozilla/4.73 [en] (X11; U; Linux 2.2.15 i686)");
curl_setopt ($ch, CURLOPT_COOKIEFILE, "/tmp/cookie.txt");
curl_setopt ($ch, CURLOPT_HEADER, 1);
curl_setopt ($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt ($ch, CURLOPT_TIMEOUT, 300);
$string = curl_exec ($ch);
curl_close($ch);
echo $string;
?>
Received on 2005-04-13