cURL / Mailing Lists / curl-library / Single Mail

curl-library

Re: pycurl/libcurl/c-ares dynamic /etc/resolv.conf

From: Dima Q <dimaqq_at_gmail.com>
Date: Wed, 26 May 2010 15:25:59 +0300

On 05/26/10 14:06, Michael Wood wrote:
> On 26 May 2010 09:27, Howard Chu<hyc_at_highlandsun.com> wrote:
>
>> Dima Q wrote:
>>
>>> On 05/26/10 01:25, Daniel Stenberg wrote:
>>>
>>>> On Tue, 25 May 2010, Dima Tisnek wrote:
>>>>
>>>>
>>>>> The long-running process tries to access the server, the lookup fails
>>>>> Later when a wireless network is discovered, I'd like to be able to
>>>>> get libcurl to talk to the server, but alas! the resolv data is
>>>>> cached somewhere and I can't get through to c-ares (or libc?) to
>>>>> re-read /etc/resolv.conf.
>>>>>
>>>> libc usally provides such a function for the stock resolver, but
>>>> c-ares doesn't.
>>>>
>>>> This is a problem that we're aware of but it has not been solved yet...
>>>>
>>>>
>>> I could come up with a patch along the lines of glibc patches (ubuntu,
>>> fedora) where system config files are stat()ed and if needed, reloaded
>>> on every init. That would only work on unix, so advice on how to guard
>>> the changes from other platforms is appreciated.
>>>
>> In my experience, it's better not to have a dynamic /etc/resolv.conf at all.
>> Instead, run a local caching DNS server and hardwire /etc/resolv.conf to
>> point to localhost. On my machines I run dnsmasq with its DBUS interface
>> enabled. I have NetworkManager patched to send the current list of DNS
>> servers to dnsmasq whenever NM connects to a network. It's a much more
>> reliable solution and no unnecessary overhead.
>>
> It sounds like he's on an embedded system, so I doubt he's using dbus.
> dnsmasq might still be an option, though. You can tell dnsmasq to
> use an arbitrary file as its resolv.conf (i.e. what upstream
> nameservers it uses), and the documentation implies that it polls this
> file (or can be made to).
>
>
I found out a workaround, I can create a short-lived curl object every
now and again and run a request to the same hostname through that. This
curl object creates new channel with c-ares, which loads new resolv.conf
and updates dns cache.

It works for the long-lived process only because I use a curl share to
cache dns and cookies.
If I didn't use curl share, I would need to cleanup and re-init the
"main" curl object.

I looked at the circumbstances where init_by_resolv_conf is called or
could be called in c-ares, and I realized that to make a proper patch I
would either need to change curl as well as change c-ares a little bit,
or change c-ares a lot.
The earlier seems very similar to what's already on curl TODO,
The latter is more robust but I doubt I could test it well enough.

Howard, I've seen similar arguments, gnu libc folks used at least since
2004 and yet modern distros (tested gentoo and ubuntu) somehow managed
to work around this, evident in strace output for curl built with glibc
resolver - there is 1 (or sometimes 2) stat64("/etc/resolv.conf") for
each request that opens a new socket. Not so with c-ares.

I think c-ares should be at least as good as glibc resolver, not worse.

Modern discussion with further references here:
http://www.eglibc.org/archives/patches/msg00772.html
-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-library
Etiquette: http://curl.haxx.se/mail/etiquette.html
Received on 2010-05-26