cURL / Mailing Lists / curl-and-php / Single Mail

curl-and-php

Re: curl not scraping Microsoft.com (something wrong?)

From: Daniel Stenberg <daniel-curl_at_haxx.se>
Date: Thu, 14 Apr 2005 08:59:04 +0200 (CEST)

On Wed, 13 Apr 2005 spamiam_at_aroint.org wrote:

> Tried that too... none of the documentation or FAQs on your website appeared
> to address this type of site. I would be happy to help write appropriate
> LiveHTTPHeaders documentation for your website if I could first figure out
> how to get it working!

I beg to differ. The info on the site explains how to deal with *any* type of
HTTP server. microsoft.com is not a different type, it just uses a particular
set of checks and requirements. Many sites are unique in that aspect.

If you repeat the sequence *exactly* as LiveHTTPHeaders reported that your
browser did, it will work.

I've done things like this with curl hundreds of times.

-- 
  Commercial curl and libcurl Technical Support: http://haxx.se/curl.html
Received on 2005-04-14