curl-and-php
Re: curl not scraping Microsoft.com (something wrong?)
From: Daniel Stenberg <daniel-curl_at_haxx.se>
Date: Thu, 14 Apr 2005 08:59:04 +0200 (CEST)
Date: Thu, 14 Apr 2005 08:59:04 +0200 (CEST)
On Wed, 13 Apr 2005 spamiam_at_aroint.org wrote:
> Tried that too... none of the documentation or FAQs on your website appeared
> to address this type of site. I would be happy to help write appropriate
> LiveHTTPHeaders documentation for your website if I could first figure out
> how to get it working!
I beg to differ. The info on the site explains how to deal with *any* type of
HTTP server. microsoft.com is not a different type, it just uses a particular
set of checks and requirements. Many sites are unique in that aspect.
If you repeat the sequence *exactly* as LiveHTTPHeaders reported that your
browser did, it will work.
I've done things like this with curl hundreds of times.
-- Commercial curl and libcurl Technical Support: http://haxx.se/curl.htmlReceived on 2005-04-14