> I'm a long-time user of cURL, but by no means a pro. I happened to get lucky that cURL "just worked" with only two options set:
> curl_setopt ($ch, CURLOPT_ENCODING, 'gzip');
> curl_setopt ($ch, CURLOPT_FILE, $fp);
> Recently, the pages I parse are coming back with an error message, though I can still open the pages in a browser and "view source".
> I've dumped headers, played with cookies and cookie jars (mostly via trial and error - I can't really get my head around the concept enough to know if I really need to be worried about cookies - I may be barking up the wrong tree entirely).
> I'm wondering if someone would be so kind as to look at http://eq.magelo.com/profile/1168428 in a browser, and advise what I need to consider to get the same content back via cURL.

I've got my fix. Probably not optimal, but I'll take what I can get!

The server was generating a 500 internal Server Error, which I was only able to get around by sending over a big header borrowed from Firefox via "LiveHTTPHeaders" as mentioned by another curl user in the mailing list archives.

Thanks to anyone that had given my problem some thought.

-------------------------------------------------------------------
List admin: http://cool.haxx.se/cgi-bin/mailman/listinfo/curl-users
FAQ: http://curl.haxx.se/docs/faq.html
Etiquette: http://curl.haxx.se/mail/etiquette.html
Received on 2009-03-31