Re: "Could not resolve host" error only on the multi interface

From: Luis Carlos Chalaca Figueira via curl-library <curl-library_at_lists.haxx.se>
Date: Wed, 1 Nov 2023 16:17:54 +0000

On 31/10/23 18:32, Dmitry Karpov via curl-library wrote:
> I would recommend to try without explicitly setting DNS server and timeout - just to use the default DNS system settings.
> Maybe, the "8.8.8.8" is not routable in your setup.
>
> Also, I would run Wireshark or tcpdump to see if your DNS queries/response were actually sent/received on the wire.
>
> Thanks,
> Dmitry Karpov
>
> -----Original Message-----
> From: curl-library <curl-library-bounces_at_lists.haxx.se> On Behalf Of Luis Carlos Chalaca Figueira via curl-library
> Sent: Tuesday, October 31, 2023 10:08 AM
> To: curl-library_at_lists.haxx.se
> Cc: Luis Carlos Chalaca Figueira <luisccfigueira_at_gmail.com>
> Subject: [EXTERNAL] "Could not resolve host" error only on the multi interface
>
> Hello everybody,
>
>
> I'm building a web crawler.
>
> Started with python but the need for speed led me to libcurl.
>
> The obstacle I'm having is: when using the multi interface a big part of the the connections cloe because of the error "Could not resolve host:"
>
> What I've tried:
>
> 1 - Confirmed that all URLs are correct.
>
> 2 - Confirmed that using the easy interface URLs are resolved.
>
> 3 - Confirmed that other programs like dig, resolvectl, nslookup can resolve URLs that multi interface couldn't.
>
> 4 - Set a "reputable" DNS server doing: curl_easy_setopt(handles[i], CURLOPT_DNS_SERVERS, "8.8.8.8");.
>
> 5 - In the file /etc/nsswitch.conf, line "hosts: files nDNS[NOTFOUNDED=return] dns" I deleted  [NOTFOUNDED=return].
>
> 6 - I've turned verbose on - Doesn't help me much.
>
> I don't know what to do more as it seems a problem inside the library, so i posted here before going digging in the library.
>
>
> System info:
>
> O.S. - Ubuntu 22.04.3 LTS
>
> Harware - HP HP Pavilion Laptop 14-ce3xxx
>
> Processor - Intel® Core™ i7-1065G7 CPU _at_ 1.30GHz × 8
>
> libcurl version - 7.81.0
>
>
> Attached I send the source code file I'm using do debug this problem.
>
>
> Best Regards,
>
> Luis Figueira
>
Dimitry,

Thanks for the response,

However it looks like my e-mail was not clear, maybe because English is
my second language.

Those 6 steps I took were made after I realized that most requests names
could not be resolved therefore they are not the problem. However I've
reset all of them to the default state.

I've used tcpdump and the output only revealed what the the system sends
packets only to one address, the stub address (Which discovered by using
nslookup). Below an exctract from the output:

19:09:50.875611 IP6 ubuntu.39274 > 2001:8a0:776c:b200::1.domain: 53886+
AAAA? e10952.a.akamaiedge.net. (41)
19:09:50.876703 IP6 ubuntu.51246 > 2001:8a0:776c:b200::1.domain: 27951+
AAAA? e-cultura.pt. (30)
19:09:50.880193 IP6 ubuntu.58635 > 2001:8a0:776c:b200::1.domain: 52688+
AAAA? orientacao-vocacional.com.pt. (46)
19:09:50.881306 IP6 2001:8a0:776c:b200::1.domain > ubuntu.47511: 24563
0/1/0 (104)
19:09:50.882880 IP6 2001:8a0:776c:b200::1.domain > ubuntu.35687: 64715
0/1/0 (110)
19:09:50.883430 IP6 2001:8a0:776c:b200::1.domain > ubuntu.43052: 35782
0/1/0 (106)

What I will do now is ask curl to give me every type of error code for
each request and see if I can get what is happening.

-- 
Cumprimentos,
Luis Figueira
-- 
Unsubscribe: https://lists.haxx.se/mailman/listinfo/curl-library
Etiquette:   https://curl.se/mail/etiquette.html

Received on 2023-11-01