Re: "Could not resolve host" error only on the multi interface

From: Dmitry Karpov via curl-library <curl-library_at_lists.haxx.se>
Date: Wed, 1 Nov 2023 18:32:05 +0000

" What I will do now is ask curl to give me every type of error code for each request and see if I can get what is happening."

I don't think libcurl can easily do that. It works with different DNS backends (i.e. c-ares or libc) and they don't provide the same error information.

In the past, I proposed to add an "info" option which would return a resolver error code, so it could be logged on remote devices and reported,
but the proposal was rejected because of differences between multiple resolver backends.

Thanks,
Dmitry

-----Original Message-----
From: Luis Carlos Chalaca Figueira <luisccfigueira_at_gmail.com>
Sent: Wednesday, November 1, 2023 9:18 AM
To: libcurl development <curl-library_at_lists.haxx.se>
Cc: Dmitry Karpov <dkarpov_at_roku.com>
Subject: [EXTERNAL] Re: "Could not resolve host" error only on the multi interface

On 31/10/23 18:32, Dmitry Karpov via curl-library wrote:
> I would recommend to try without explicitly setting DNS server and timeout - just to use the default DNS system settings.
> Maybe, the "8.8.8.8" is not routable in your setup.
>
> Also, I would run Wireshark or tcpdump to see if your DNS queries/response were actually sent/received on the wire.
>
> Thanks,
> Dmitry Karpov
>
> -----Original Message-----
> From: curl-library <curl-library-bounces_at_lists.haxx.se> On Behalf Of
> Luis Carlos Chalaca Figueira via curl-library
> Sent: Tuesday, October 31, 2023 10:08 AM
> To: curl-library_at_lists.haxx.se
> Cc: Luis Carlos Chalaca Figueira <luisccfigueira_at_gmail.com>
> Subject: [EXTERNAL] "Could not resolve host" error only on the multi
> interface
>
> Hello everybody,
>
>
> I'm building a web crawler.
>
> Started with python but the need for speed led me to libcurl.
>
> The obstacle I'm having is: when using the multi interface a big part of the the connections cloe because of the error "Could not resolve host:"
>
> What I've tried:
>
> 1 - Confirmed that all URLs are correct.
>
> 2 - Confirmed that using the easy interface URLs are resolved.
>
> 3 - Confirmed that other programs like dig, resolvectl, nslookup can resolve URLs that multi interface couldn't.
>
> 4 - Set a "reputable" DNS server doing: curl_easy_setopt(handles[i], CURLOPT_DNS_SERVERS, "8.8.8.8");.
>
> 5 - In the file /etc/nsswitch.conf, line "hosts: files nDNS[NOTFOUNDED=return] dns" I deleted  [NOTFOUNDED=return].
>
> 6 - I've turned verbose on - Doesn't help me much.
>
> I don't know what to do more as it seems a problem inside the library, so i posted here before going digging in the library.
>
>
> System info:
>
> O.S. - Ubuntu 22.04.3 LTS
>
> Harware - HP HP Pavilion Laptop 14-ce3xxx
>
> Processor - Intel® Core™ i7-1065G7 CPU _at_ 1.30GHz × 8
>
> libcurl version - 7.81.0
>
>
> Attached I send the source code file I'm using do debug this problem.
>
>
> Best Regards,
>
> Luis Figueira
>
Dimitry,

Thanks for the response,

However it looks like my e-mail was not clear, maybe because English is my second language.

Those 6 steps I took were made after I realized that most requests names could not be resolved therefore they are not the problem. However I've reset all of them to the default state.

I've used tcpdump and the output only revealed what the the system sends packets only to one address, the stub address (Which discovered by using nslookup). Below an exctract from the output:

19:09:50.875611 IP6 ubuntu.39274 > 2001:8a0:776c:b200::1.domain: 53886+ AAAA? e10952.a.akamaiedge.net. (41)
19:09:50.876703 IP6 ubuntu.51246 > 2001:8a0:776c:b200::1.domain: 27951+ AAAA? e-cultura.pt. (30)
19:09:50.880193 IP6 ubuntu.58635 > 2001:8a0:776c:b200::1.domain: 52688+ AAAA? orientacao-vocacional.com.pt. (46)
19:09:50.881306 IP6 2001:8a0:776c:b200::1.domain > ubuntu.47511: 24563
0/1/0 (104)
19:09:50.882880 IP6 2001:8a0:776c:b200::1.domain > ubuntu.35687: 64715
0/1/0 (110)
19:09:50.883430 IP6 2001:8a0:776c:b200::1.domain > ubuntu.43052: 35782
0/1/0 (106)

What I will do now is ask curl to give me every type of error code for each request and see if I can get what is happening.

--
Cumprimentos,
Luis Figueira

-- 
Unsubscribe: https://lists.haxx.se/mailman/listinfo/curl-library
Etiquette:   https://curl.se/mail/etiquette.html

Received on 2023-11-01