Buy commercial curl support from WolfSSL. We help you work
out your issues, debug your libcurl applications, use the API, port to new
platforms, add new features and more. With a team lead by the curl founder
himself.
Re: "Could not resolve host" error only on the multi interface
- Contemporary messages sorted: [ by date ] [ by thread ] [ by subject ] [ by author ] [ by messages with attachments ]
From: Luis Carlos Chalaca Figueira via curl-library <curl-library_at_lists.haxx.se>
Date: Wed, 1 Nov 2023 18:33:10 +0000
On 31/10/23 19:25, Jeffrey Walton wrote:
> On Tue, Oct 31, 2023 at 1:28 PM Luis Carlos Chalaca Figueira via
> curl-library <curl-library_at_lists.haxx.se> wrote:
>> I'm building a web crawler.
>>
>> Started with python but the need for speed led me to libcurl.
>>
>> The obstacle I'm having is: when using the multi interface a big part of
>> the the connections cloe because of the error "Could not resolve host:"
>>
>> What I've tried:
>>
>> 1 - Confirmed that all URLs are correct.
>>
>> 2 - Confirmed that using the easy interface URLs are resolved.
>>
>> 3 - Confirmed that other programs like dig, resolvectl, nslookup can
>> resolve URLs that multi interface couldn't.
>>
>> 4 - Set a "reputable" DNS server doing: curl_easy_setopt(handles[i],
>> CURLOPT_DNS_SERVERS, "8.8.8.8");.
>>
>> 5 - In the file /etc/nsswitch.conf, line "hosts: files
>> nDNS[NOTFOUNDED=return] dns" I deleted [NOTFOUNDED=return].
>>
>> 6 - I've turned verbose on - Doesn't help me much.
>>
>> I don't know what to do more as it seems a problem inside the library,
>> so i posted here before going digging in the library.
> I _think_ step (5) - modification of /etc/nsswitch.conf - may be the
> problem. I seem to recall '[NOTFOUNDED=return]' is needed to ensure
> the next provider is tried. Without it, weird things happen.
>
> I don't have much experience with fiddling with nsswitch.conf. But I
> vaguely recall reading about it several times.
>
> And as
>
> Dmitry said, Wireshark may be a good idea. That should tell you if nss
> is making the DNS query. It may show only the mDNS query is happening.
>
> Jeff
Jeff,
Thanks for the response,
All those steps I did after the errors, therefore they are not the cause
(it looks my English was not clear , I'm sorry).
Anyway I'm looking now at wireshark and it looks a great program to
debug networking issues.
After more testing it looks like it may be caused by the use of threads
since requesting only in one thread does not show that error.
Another question:
Since I'm new to this mailing list and I've never been in a mailing list
I don't know if should respond to every e-mail that was sent in response
to mine or should I respond as if I'm responding in a forum and in one
e-mail can respond to all the people that responded to me?
Thanks for your help.
Date: Wed, 1 Nov 2023 18:33:10 +0000
On 31/10/23 19:25, Jeffrey Walton wrote:
> On Tue, Oct 31, 2023 at 1:28 PM Luis Carlos Chalaca Figueira via
> curl-library <curl-library_at_lists.haxx.se> wrote:
>> I'm building a web crawler.
>>
>> Started with python but the need for speed led me to libcurl.
>>
>> The obstacle I'm having is: when using the multi interface a big part of
>> the the connections cloe because of the error "Could not resolve host:"
>>
>> What I've tried:
>>
>> 1 - Confirmed that all URLs are correct.
>>
>> 2 - Confirmed that using the easy interface URLs are resolved.
>>
>> 3 - Confirmed that other programs like dig, resolvectl, nslookup can
>> resolve URLs that multi interface couldn't.
>>
>> 4 - Set a "reputable" DNS server doing: curl_easy_setopt(handles[i],
>> CURLOPT_DNS_SERVERS, "8.8.8.8");.
>>
>> 5 - In the file /etc/nsswitch.conf, line "hosts: files
>> nDNS[NOTFOUNDED=return] dns" I deleted [NOTFOUNDED=return].
>>
>> 6 - I've turned verbose on - Doesn't help me much.
>>
>> I don't know what to do more as it seems a problem inside the library,
>> so i posted here before going digging in the library.
> I _think_ step (5) - modification of /etc/nsswitch.conf - may be the
> problem. I seem to recall '[NOTFOUNDED=return]' is needed to ensure
> the next provider is tried. Without it, weird things happen.
>
> I don't have much experience with fiddling with nsswitch.conf. But I
> vaguely recall reading about it several times.
>
> And as
>
> Dmitry said, Wireshark may be a good idea. That should tell you if nss
> is making the DNS query. It may show only the mDNS query is happening.
>
> Jeff
Jeff,
Thanks for the response,
All those steps I did after the errors, therefore they are not the cause
(it looks my English was not clear , I'm sorry).
Anyway I'm looking now at wireshark and it looks a great program to
debug networking issues.
After more testing it looks like it may be caused by the use of threads
since requesting only in one thread does not show that error.
Another question:
Since I'm new to this mailing list and I've never been in a mailing list
I don't know if should respond to every e-mail that was sent in response
to mine or should I respond as if I'm responding in a forum and in one
e-mail can respond to all the people that responded to me?
Thanks for your help.
-- Cumprimentos, Luis Figueira -- Unsubscribe: https://lists.haxx.se/mailman/listinfo/curl-library Etiquette: https://curl.se/mail/etiquette.htmlReceived on 2023-11-01