On Thu, Aug 07, 2008 at 01:02:59PM -0600, Doug McNutt wrote:
> I noticed yesterday that my scripted calls to the curl tool generate multiple requests to BIND for the same domain name as I work my way down into a site that I can't get to while bypassing the login shenanigans.
>
> Has anyone done the work required to manage a cache of IP addresses to prevent that? A daemon process that preprocesses calls to curl that would otherwise create a new process comes to mind.

Yes, someone has implemented this, but not for curl specifically. nscd is
a daemon that does exactly what you propose and is available as a part
of glibc.

> Does CurlLib have a capability like that that might push me over there? (Yeah, I probably ought to do that anyway.)

libcurl maintains a DNS cache, but, naturally, it's not persistent between
invocations of the curl tool.

> I could "dig" for the addresses and store them but I'm not so sure how to handle errors that would occur when things change. Does the curl tool return anything helpful for a second try when a dotted decimal address doesn't work? Can I expect web sites to complain when they see their addresses in dotted decimal rather than as a domain name?

Yes, virtual web hosts will complain, and those are probably in the majority
these days.

> How do web browsers handle their caching when an anchor points to a relative URL? Would I need to provide a replacement header line with the domain name when I provide a URL using dotted decimal?

A relative URL has to be converted at some level into an absolute one before
it can be dereferenced, and that doesn't affect the DNS caching algorithm.

>>> Dan

-- 
http://www.MoveAnnouncer.com              The web change of address service
          Let webmasters know that your web site has moved
-------------------------------------------------------------------
List admin: http://cool.haxx.se/cgi-bin/mailman/listinfo/curl-users
FAQ:        http://curl.haxx.se/docs/faq.html
Etiquette:  http://curl.haxx.se/mail/etiquette.html