curl-and-php
Re: CURLE_COULDNT_CONNECT on some valid sites
From: Massimo Lombardo <unwiredbrain_at_gmail.com>
Date: Thu, 18 Jun 2009 19:14:23 +0200
Date: Thu, 18 Jun 2009 19:14:23 +0200
> I have a list of (~75000) sites that I need to check the status of.
Are you using the multi interface?
If not, that would solve TONS of problems if set up properly (i.e.
using curl_multi_select & stuff)
> [pretend there's that huge list of options here]
Is this huge list of options really necessary?
Why are you trying to appear like the GoogleBot?
Maybe some sites reject you because you don't have GoogleBot's IP!
And why are you forcing HTTP/1.0?
-- Massimo Lombardo Linux user #437712 _______________________________________________ http://cool.haxx.se/cgi-bin/mailman/listinfo/curl-and-phpReceived on 2009-06-18