On Tue, 1 Nov 2005, Russell Neufeld wrote:

> We are using DNS round-robining to load balance among servers and we're
> having a hard time getting curl to obey it. (Fedora Core 4, libcurl/7.13.1)
> I have a name - www.cluster.archivas.com - which resolves to one of
> 192.168.139.103, 192.168.139.104, 192.168.139.105, or 192.168.139.106.
> Running "hosts www.cluster.archivas.com" multiple times over and over shows
> that the ip addresses are round robining and returned in different order
> each time. However running the command line "curl
> http://www.cluster.archivas.com/file" multiple times over and over always
> hits the same server. (If I wait about a minute and try again, it might
> choose another server.)

What glibc version are you using? Is libcurl built ipv6 enabled?

I've noticed something similar before and narrowing in the details could help.

> The interesting thing is that other network utilities do respect the
> round-robining. Running wget with the same URL over and over does round
> robin, as does repeated invokations of ping, so I feel like I've eliminated
> any OS name caching. Named is not running on the machine.
> I can imagine that I'm not setting the right curl opt when I'm using
> pycurl or libcurl, but I don't understand why the command line curl would
> exhibit the same behavior.

curl _should_ exhibit and use the same round-robin approach like the rest. It
simply asks for a set of addresses for the given name and they are supposed to
be returned in a "random" fashion. curl (or rather libcurl) simply tries the
addresses in the same order it gets them.

When I noticed a slight problem related to this the other day
(http://curl.haxx.se/mail/lib-2005-10/0100.html) I found out that libcurl
behaved differently when I built it with IPv6 support disabled vs enabled...

-- 
  Commercial curl and libcurl Technical Support: http://haxx.se/curl.html