cURL / Mailing Lists / curl-and-php / Single Mail

curl-and-php

Re: curl not scraping Microsoft.com (something wrong?)

From: Daniel Stenberg <daniel-curl_at_haxx.se>
Date: Thu, 14 Apr 2005 16:20:56 +0200 (CEST)

On Thu, 14 Apr 2005 spamiam_at_aroint.org wrote:

> Perhaps I missed where on your website it explains precisely *what* needs to
> be copied from the headers

You should make sure that all headers your curl usage produce look like the
ones you see in the header capture. With as few exceptions as possible.

> and *how* to do it in curl in a step-by-step fashion.

Well, this is a start:

         http://curl.haxx.se/docs/httpscripting.html

And more libcurl-oriented tuturial stuff is here:

         http://curl.haxx.se/libcurl/c/libcurl-tutorial.html

But I'm not a PHP guy so don't expect PHP-related docs from my end.

> I'm not expecting a cookie cutter solution, but it shouldn't be some
> mystical process, either.

It is far from "mystical". You send a HTTP request and headers, and you get
back HTTP headers and a response. It is very simple.

All info about them is found in RFC2616.

-- 
  Commercial curl and libcurl Technical Support: http://haxx.se/curl.html
Received on 2005-04-14