curl / Mailing Lists / curl-library / Single Mail
Buy commercial curl support from WolfSSL. We help you work out your issues, debug your libcurl applications, use the API, port to new platforms, add new features and more. With a team lead by the curl founder himself.

Re: "Could not resolve host" error only on the multi interface

From: Jeffrey Walton via curl-library <curl-library_at_lists.haxx.se>
Date: Tue, 31 Oct 2023 15:25:19 -0400

On Tue, Oct 31, 2023 at 1:28 PM Luis Carlos Chalaca Figueira via
curl-library <curl-library_at_lists.haxx.se> wrote:
>
> I'm building a web crawler.
>
> Started with python but the need for speed led me to libcurl.
>
> The obstacle I'm having is: when using the multi interface a big part of
> the the connections cloe because of the error "Could not resolve host:"
>
> What I've tried:
>
> 1 - Confirmed that all URLs are correct.
>
> 2 - Confirmed that using the easy interface URLs are resolved.
>
> 3 - Confirmed that other programs like dig, resolvectl, nslookup can
> resolve URLs that multi interface couldn't.
>
> 4 - Set a "reputable" DNS server doing: curl_easy_setopt(handles[i],
> CURLOPT_DNS_SERVERS, "8.8.8.8");.
>
> 5 - In the file /etc/nsswitch.conf, line "hosts: files
> nDNS[NOTFOUNDED=return] dns" I deleted [NOTFOUNDED=return].
>
> 6 - I've turned verbose on - Doesn't help me much.
>
> I don't know what to do more as it seems a problem inside the library,
> so i posted here before going digging in the library.

I _think_ step (5) - modification of /etc/nsswitch.conf - may be the
problem. I seem to recall '[NOTFOUNDED=return]' is needed to ensure
the next provider is tried. Without it, weird things happen.

I don't have much experience with fiddling with nsswitch.conf. But I
vaguely recall reading about it several times.

And as

Dmitry said, Wireshark may be a good idea. That should tell you if nss
is making the DNS query. It may show only the mDNS query is happening.

Jeff
-- 
Unsubscribe: https://lists.haxx.se/mailman/listinfo/curl-library
Etiquette:   https://curl.se/mail/etiquette.html
Received on 2023-10-31