cURL / Mailing Lists / curl-library / Single Mail


Re: Fwd: Duplicate DNS Requests using cURL and C-ARES

From: Dave Reisner <>
Date: Thu, 27 Nov 2014 12:11:20 -0500

On Thu, Nov 27, 2014 at 12:01:24PM +0000, Josh Russell wrote:
> Hi All,
> I'm working on an application which uses a combination of libcurl 7.36.0 and
> c-ares 1.10.1 (c-ares for async DSN lookup). The application is cross compiled
> for various platforms but is closely POSIX like.
> I'm using curl_easy_init & curl_easy_setopt to set up my curl handles. Those
> handles are then processed using curl_multi but I'm also using curl_share
> What I'm finding is that when I make a few requests to various URLs on the same
> domain multiple DNS requests are made for the same domain. Once the first DNS
> response returned its cached I don't see any more requests for that domain - as
> expected.
> The problem is, if I make enough requests to the same domain before the first
> DNS response returns, I can my client banned by the DNS server if I make too
> many requests.
> Is there any way to de-duplicate concurrent requests for the same DNS address?

I'd be interested in a fix for this as well -- it seems that the DNS
cache has no protection against stampedes when using asynchronous
resolution. This behavior occurs for the threaded resolver, not just
c-ares. Here's a minimal example which can be used to show the

When run with the threaded resolver, strace shows a DNS request for each
handle, i.e. (output trimmed):

$ strace -fe sendmmsg ./resolv
[pid 3536] sendmmsg(5, {{{msg_name(0)=NULL, msg_iov(1)=[{"w}\1\0\0\1\0\0\0\0\0\0\3www\6google\3com\0\0\1\0\1", ...
[pid 3535] sendmmsg(3, {{{msg_name(0)=NULL, msg_iov(1)=[{"\303\207\1\0\0\1\0\0\0\0\0\0\3www\6google\3com\0\0\1\0\1", ...
[pid 3534] sendmmsg(4, {{{msg_name(0)=NULL, msg_iov(1)=[{"\367+\1\0\0\1\0\0\0\0\0\0\3www\6google\3com\0\0\1\0\1", ...
[pid 3533] sendmmsg(6, {{{msg_name(0)=NULL, msg_iov(1)=[{"\313\4\1\0\0\1\0\0\0\0\0\0\3www\6google\3com\0\0\1\0\1", ...

And when preloading a which is built with the vanilla system
resolver, it's just a single request:

$ LD_PRELOAD=~/code/curl/lib/.libs/ strace -e sendmmsg ./resolv
sendmmsg(3, {{{msg_name(0)=NULL, msg_iov(1)=[{"\21\37\1\0\0\1\0\0\0\0\0\0\3www\6google\3com\0\0\1\0\1", ...

I'm currently working on porting an application using pthreads and
individual easy handles to using the multi interface and was surprised
to see this behavior. It actually becomes favorable to use synchronous
resolution when many handles are added at once and all connect to the
same domain.

List admin:
Received on 2014-11-27