Upgrading curl from 7.86.0 to 8.0.1 causing a lot of logs shown "Connection #xxx is still name resolving, can't reuse" #11394
Comments
|
@icing Could you explain this problem? Thank you! |
|
I see no verbose output from 10-at-a-time. Have you modified it further than changing the URLs? I am having trouble reproducing this. The message you mention is only emitted when an async resolver is configured, right? My question is then, will existing connections be correctly reused or is a new connection opened for every URL as long as the resolver does not return? |
|
Thank you for your reply. Blow is test example source code. #include <errno.h>
#include <stdlib.h>
#include <string>
#include <iostream>
#ifndef WIN32
#include <unistd.h>
#endif
#include <curl/curl.h>
static const char* urls[] = {
"https://www.man7.org/linux/man-pages/man2/socket.2.html",
"https://www.man7.org/linux/man-pages/man2/accept.2.html",
"https://www.man7.org/linux/man-pages/man2/bind.2.html",
"https://www.man7.org/linux/man-pages/man2/getsockname.2.html",
"https://www.man7.org/linux/man-pages/man3/sd_is_fifo.3.html",
"https://www.man7.org/linux/man-pages/man2/getpeername.2.html",
"https://www.man7.org/linux/man-pages/man7/ip.7.html",
"https://www.man7.org/linux/man-pages/man7/udp.7.html",
"https://www.man7.org/linux/man-pages/man2/sendmsg.2.html",
"https://www.man7.org/linux/man-pages/man2/write.2.html",
"https://www.man7.org/linux/man-pages/man2/lseek.2.html",
};
#define MAX_PARALLEL 10L /* number of simultaneous transfers */
#define NUM_URLS sizeof(urls)/sizeof(char *)
#define MAX_LIBCURL_LOG_DATA_SIZE (1024*4)
int my_curl_debug_callback(CURL* handle, /* the handle/transfer this concerns */
curl_infotype type, /* what kind of data */
char* data, /* points to the data */
size_t size, /* size of the data pointed to */
void* userptr) /* whatever the user please */
{
if (size <= MAX_LIBCURL_LOG_DATA_SIZE)
{
std::string str(data, size);
std::cout << str << std::endl;
}
return 0;
}
static size_t write_cb(char* data, size_t n, size_t l, void* userp)
{
/* take care of the data here, ignored in this example */
(void)data;
(void)userp;
return n * l;
}
static void add_transfer(CURLM* cm, int i, int* left)
{
CURL* eh = curl_easy_init();
curl_easy_setopt(eh, CURLOPT_WRITEFUNCTION, write_cb);
curl_easy_setopt(eh, CURLOPT_URL, urls[i]);
curl_easy_setopt(eh, CURLOPT_PRIVATE, urls[i]);
curl_easy_setopt(eh, CURLOPT_SSL_VERIFYPEER, 0L);
curl_easy_setopt(eh, CURLOPT_VERBOSE, 1L);
curl_easy_setopt(eh, CURLOPT_DEBUGFUNCTION, my_curl_debug_callback);
curl_multi_add_handle(cm, eh);
(*left)++;
}
int main(void)
{
CURLM* cm;
CURLMsg* msg;
unsigned int transfers = 0;
int msgs_left = -1;
int left = 0;
curl_global_init(CURL_GLOBAL_ALL);
cm = curl_multi_init();
/* Limit the amount of simultaneous connections curl should allow: */
curl_multi_setopt(cm, CURLMOPT_MAXCONNECTS, (long)MAX_PARALLEL);
for (transfers = 0; transfers < MAX_PARALLEL && transfers < NUM_URLS;
transfers++)
add_transfer(cm, transfers, &left);
do {
int still_alive = 1;
curl_multi_perform(cm, &still_alive);
while ((msg = curl_multi_info_read(cm, &msgs_left))) {
if (msg->msg == CURLMSG_DONE) {
char* url;
CURL* e = msg->easy_handle;
curl_easy_getinfo(msg->easy_handle, CURLINFO_PRIVATE, &url);
fprintf(stderr, "R: %d - %s <%s>\n",
msg->data.result, curl_easy_strerror(msg->data.result), url);
curl_multi_remove_handle(cm, e);
curl_easy_cleanup(e);
left--;
}
else {
fprintf(stderr, "E: CURLMsg (%d)\n", msg->msg);
}
if (transfers < NUM_URLS)
add_transfer(cm, transfers++, &left);
}
if (left)
curl_multi_wait(cm, NULL, 0, 1000, NULL);
} while (left);
curl_multi_cleanup(cm);
curl_global_cleanup();
return EXIT_SUCCESS;
}We configured async resolver. A completed conn can be reused. But if you add multiple easy handles belonging to the same host to a multi handle, and then execute curl_multi_perform(), a large number of "Connection #xxx is still name resolving, can't reuse" will appear. |
|
Maybe this is a normal behavior, but would like to get your confirmation. |
|
I think this verbose output should not happen in your example. We should change this. The change you referenced was necessary if the application uses But you are not using Let's see what @bagder has to say to this when he is back. |
|
Thank you for your explanation. |
The message does not help and might get spewed a lot during times. Reported-by: yushicheng7788 on github Fixes #11394
The message does not help and might get spewed a lot during times. Reported-by: yushicheng7788 on github Fixes curl#11394 Closes curl#11529
I did this
I ran the example code https://curl.se/libcurl/c/10-at-a-time.html and replaced the urls array in the code with urls from the same host. There are a lot of logs showing "connection #xxx is still name resolving, can't reuse". After a series of investigations, I found that this commit #10456 caused it. Is this a normal phenomenon normal? It seems that this change will cause conn->bits.close to be false, so the
if (check->connect_only || check->bits.close)of the ConnectionExists function is judged to be false, causing the function to continue to execute, and a large number of logs appear " Connection #xxx is still name resolving, can't reuse". If the change is rolled back, there will be no log indicating that the connection cannot be reused.I expected the following
I would like to know whether this phenomenon is normal or if there is an error.
curl/libcurl version
8.0.1
operating system
Mac / Windows
The text was updated successfully, but these errors were encountered: