On Wed, Apr 23, 2008 at 04:20:27PM -0300, Arthur Rodrigues wrote:
> I'm running a multi-threaded webCrawler using Curl, but eventually, after
> downloading about 600 sites
> it starts return error 6 or error 23 even for existent sites. I'm working on
> Gentoo 2.16.2.
>
> For those sites returning error, I easy access them via browser.
>
> Do you have any ideas of what might be happening?

Could it be that the process is running out of free sockets to use for the
DNS resolver? Could it be a bona-fide temporary problem on the remote
site's DNS server, or a temporary network problem? Does it work on such
sites if you try it again? Are you using c-ares? Error 23 indicates an
error code returned by the app and is likely a separate problem.

>>> Dan

-- 
http://www.MoveAnnouncer.com              The web change of address service
          Let webmasters know that your web site has moved