On Wed, Nov 26, 2008 at 1:18 PM, Daniel Stenberg <daniel_at_haxx.se> wrote:
> On Wed, 26 Nov 2008, Tsachi wrote:
>
>> Now, many of the urls I am fetching belongs to the same domain.
>> Example: www.aaa.com , www.aaa.com/1/ , www.aaa.com/2/
>>
>> I can see that all DNS requests are sent at the beginning (probably right
>> after calling curl_multi_perform()) that's b/c I am using asynchronous name
>> resolves.
>
> And you're fetching these pages in parallell, right?

Yep

>> I have noticed that the libcurl issues a different dns request for every
>> URL in the multi handle although many of them share the same domain. This
>> means that for the example above I get to see 3 DNS requests for
>> www.aaa.com.
>
> libcurl has an internal DNS cache that by default is used for all easy
> handles within a multi handle, but that of course requires that the host
> name has already been resolved fine at the time a new connection is about to
> be setup.
>
>> I am wondering if there is an option that I can use to tell the libcurl to
>> issue only one DNS request for www.aaa.com and only when response arrives
>> perform the HTTP requests for all the URL's sharing the same domain ?
>
> There's no such option. But you could delay the following requests to not
> get started until the first one delivers data as then you know it has been
> resolved and will be dealt with by the cache...
>
> --
>
> / daniel.haxx.se
>

Thanks, that is what I will probably do.

Is there a way to know that I got a DNS response before the actual
http connection starts (callback / info) ?
This way I could add the others easy handles (with the same domain
URLs) at this point.

Thanks,
Tsachi
Received on 2008-11-26