Buy commercial curl support from WolfSSL. We help you work
out your issues, debug your libcurl applications, use the API, port to new
platforms, add new features and more. With a team lead by the curl founder
himself.
Re: curl-multi-socket interface too slow
- Contemporary messages sorted: [ by date ] [ by thread ] [ by subject ] [ by author ] [ by messages with attachments ]
From: Henrik Holst via curl-library <curl-library_at_lists.haxx.se>
Date: Wed, 1 Jun 2022 21:02:07 +0200
this part also doesn't look right to me:
if(events[i].events & EPOLLOUT) {
for (int j = 0; j < MAX_CONNECTIONS; j++)
{
if (events[i].data.fd == sockfd[j])
{
index = j;
break;
}
}
snprintf(get_buffer[index], sizeof(get_buffer[index]),
"GET %s HTTP/1.1\r\nHost: %s\r\nUser-Agent:
Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko)
Chrome/51.0.2704.103 Safari/537.36\r\n\r\n", "/", domains[i]);
ssize_t nByte = 0;
if (buffer_used[index] < strlen(get_buffer[index]))
nByte = send_data(index);
if (nByte > 0)
{
buffer_used[index] += nByte;
total_bytes += nByte;
}
if (nByte == -1 && errno == EPIPE)
{
get_domain_and_ip(index);
}
}
If I understand the code correctly then sockfd[j] here is first the
dns resolver socket?! and when send to it fails you call
get_domain_and_ip which
connects to the actual server and you do this on EPOLLOUT which imho
is the wrong place. EPOLLOUT will only be signalled if the socket have
become writable
after send have returned EAGAIN to signal that it would be blocking.
By pure luck it also is being sent along whenever EPOLLIN is signalled
if the socket is in
a writable state at the time.
Also I don't think the linear search for the socket doesn't help
either, better to store the index in the event and use that instead of
the fd.
/HH
Den ons 1 juni 2022 kl 20:42 skrev Henrik Holst <
henrik.holst_at_millistream.com>:
> I wonder if you in that specific example (the stackexchange one) are
> hitting some limit in the dns resolver since "domains/s=161.98" which is
> the max value there would be something like (1 event for dns reply, 1 for
> connect, 1 for write is what I'm guesstimating at here) ~486 events in
> epoll per second is not much events at all.
>
> I wouldn't be at all surprised if public DNS servers such as the
> Cloudflare and Google one used in your example have some kind of rate
> limiting.
>
> /HH
>
> Den ons 1 juni 2022 kl 19:42 skrev James Read via curl-library <
> curl-library_at_lists.haxx.se>:
>
>>
>>
>> On Wed, Jun 1, 2022 at 2:22 PM Aleh Linkin <aleh.linkin_at_idt.net> wrote:
>>
>>> Thanks for the update. Could you be a bit more specific? What do you
>>> mean under "problem with the TCP/IP stack"? If I'd like to use
>>> curl_multi_socket_action I had to sacrifice some CPU?
>>>
>>
>> Some time ago I posted a question related on stackoverflow:
>> https://stackoverflow.com/questions/70584121/why-doesnt-my-epoll-based-program-improve-performance-by-increasing-the-number
>>
>>
>> The question includes no cURL based code which demonstrates the
>> limitations of epoll with the current TCP/IP stack. I don't know exactly
>> what the problem with the stack is. I'm still working on it.
>>
>> James Read
>>
>>
>>>
>>> On Wed, Jun 1, 2022 at 4:04 PM James Read <jamesread5737_at_gmail.com>
>>> wrote:
>>>
>>>> The problem is not with cURL. I have also run into these problems using
>>>> cURL. I recoded using sockets and epoll and got the same results. The
>>>> problem is with the TCP/IP stack.
>>>>
>>>> On Wed, Jun 1, 2022 at 1:46 PM Aleh Linkin via curl-library <
>>>> curl-library_at_lists.haxx.se> wrote:
>>>>
>>>>> 1. easy_handles
>>>>> ```
>>>>> SET_OPT(CURLOPT_ERRORBUFFER, m_errorBuffer);
>>>>> setVerbose(true);
>>>>> setUrl(url);
>>>>> setTimeout(timeout);
>>>>> setConnectTimeout(connectTimeout);
>>>>> setProxy(proxy);
>>>>> setProxyPort(proxyPort);
>>>>> setHttpHeaders(httpHeaders);
>>>>> SET_OPT(CURLOPT_WRITEFUNCTION, writeFunction);
>>>>> SET_OPT(CURLOPT_WRITEDATA, m_requestState);
>>>>> SET_OPT(CURLOPT_DEBUGFUNCTION, CurlRequest::debugFunction);
>>>>> SET_OPT(CURLOPT_DEBUGDATA, this);
>>>>> SET_OPT(CURLOPT_PRIVATE, this);
>>>>> SET_OPT(CURLOPT_NOSIGNAL, 1L);
>>>>> ```
>>>>>
>>>>> 2. HTTPS. TLS based on NSS SSL backend
>>>>>
>>>>> 3. Yes. I've used perf and valgrind-cachgrind but I can't take the
>>>>> reason from results. For example perf part:
>>>>> ```
>>>>> Samples: 1K of event 'cycles:uppp', Event count (approx.): 9397031894
>>>>> Children Self Command Shared Object Symbol
>>>>> - 23.25% 2.43% lecappqry.1 [unknown] [.]
>>>>> 0000000000000000
>>>>> - 21.58% 0
>>>>> 5.29% _int_malloc
>>>>> 1.42% free
>>>>> 1.13% Curl_http
>>>>> 1.07% _int_free
>>>>> + 0.88% __clock_gettime
>>>>> 0.72% _IO_vsnprintf
>>>>> 0.71% __libc_calloc
>>>>> + 5.07% 0.00% lecappqry.1 [unknown] [k]
>>>>> 0x0000000000667849
>>>>> + 5.07% 0.00% lecappqry.1 [unknown] [k]
>>>>> 0x00007f33507ad900
>>>>> + 5.07% 4.87% lecappqry.1 lecappqry_grpc [.]
>>>>> dprintf_formatf
>>>>> + 4.88% 4.71% lecappqry.1 libc-2.17.so [.] _int_free
>>>>> ```
>>>>>
>>>>> On Mon, May 30, 2022 at 9:07 PM Patrick Schlangen <
>>>>> patrick_at_schlangen.me> wrote:
>>>>>
>>>>>> Am 30.05.2022 um 16:23 schrieb Aleh Linkin via curl-library <
>>>>>> curl-library_at_lists.haxx.se>:
>>>>>> >
>>>>>> > I understand. Thank you.
>>>>>> > But why multi_socket_action eats about twice time more CPU then
>>>>>> curl_multi_perform called in cycle with 10ms pause?
>>>>>>
>>>>>> Can you supply some more details about how the easy handle is
>>>>>> configured and the type of transfers you are running? Is it HTTP? Are you
>>>>>> using TLS? Did you use perf or a similar profiler yet to see where time is
>>>>>> spent?
>>>>>>
>>>>>> Thanks,
>>>>>>
>>>>>> Patrick
>>>>>>
>>>>>> --
>>>>> Unsubscribe: https://lists.haxx.se/listinfo/curl-library
>>>>> Etiquette: https://curl.haxx.se/mail/etiquette.html
>>>>>
>>>> --
>> Unsubscribe: https://lists.haxx.se/listinfo/curl-library
>> Etiquette: https://curl.haxx.se/mail/etiquette.html
>>
>
Date: Wed, 1 Jun 2022 21:02:07 +0200
this part also doesn't look right to me:
if(events[i].events & EPOLLOUT) {
for (int j = 0; j < MAX_CONNECTIONS; j++)
{
if (events[i].data.fd == sockfd[j])
{
index = j;
break;
}
}
snprintf(get_buffer[index], sizeof(get_buffer[index]),
"GET %s HTTP/1.1\r\nHost: %s\r\nUser-Agent:
Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko)
Chrome/51.0.2704.103 Safari/537.36\r\n\r\n", "/", domains[i]);
ssize_t nByte = 0;
if (buffer_used[index] < strlen(get_buffer[index]))
nByte = send_data(index);
if (nByte > 0)
{
buffer_used[index] += nByte;
total_bytes += nByte;
}
if (nByte == -1 && errno == EPIPE)
{
get_domain_and_ip(index);
}
}
If I understand the code correctly then sockfd[j] here is first the
dns resolver socket?! and when send to it fails you call
get_domain_and_ip which
connects to the actual server and you do this on EPOLLOUT which imho
is the wrong place. EPOLLOUT will only be signalled if the socket have
become writable
after send have returned EAGAIN to signal that it would be blocking.
By pure luck it also is being sent along whenever EPOLLIN is signalled
if the socket is in
a writable state at the time.
Also I don't think the linear search for the socket doesn't help
either, better to store the index in the event and use that instead of
the fd.
/HH
Den ons 1 juni 2022 kl 20:42 skrev Henrik Holst <
henrik.holst_at_millistream.com>:
> I wonder if you in that specific example (the stackexchange one) are
> hitting some limit in the dns resolver since "domains/s=161.98" which is
> the max value there would be something like (1 event for dns reply, 1 for
> connect, 1 for write is what I'm guesstimating at here) ~486 events in
> epoll per second is not much events at all.
>
> I wouldn't be at all surprised if public DNS servers such as the
> Cloudflare and Google one used in your example have some kind of rate
> limiting.
>
> /HH
>
> Den ons 1 juni 2022 kl 19:42 skrev James Read via curl-library <
> curl-library_at_lists.haxx.se>:
>
>>
>>
>> On Wed, Jun 1, 2022 at 2:22 PM Aleh Linkin <aleh.linkin_at_idt.net> wrote:
>>
>>> Thanks for the update. Could you be a bit more specific? What do you
>>> mean under "problem with the TCP/IP stack"? If I'd like to use
>>> curl_multi_socket_action I had to sacrifice some CPU?
>>>
>>
>> Some time ago I posted a question related on stackoverflow:
>> https://stackoverflow.com/questions/70584121/why-doesnt-my-epoll-based-program-improve-performance-by-increasing-the-number
>>
>>
>> The question includes no cURL based code which demonstrates the
>> limitations of epoll with the current TCP/IP stack. I don't know exactly
>> what the problem with the stack is. I'm still working on it.
>>
>> James Read
>>
>>
>>>
>>> On Wed, Jun 1, 2022 at 4:04 PM James Read <jamesread5737_at_gmail.com>
>>> wrote:
>>>
>>>> The problem is not with cURL. I have also run into these problems using
>>>> cURL. I recoded using sockets and epoll and got the same results. The
>>>> problem is with the TCP/IP stack.
>>>>
>>>> On Wed, Jun 1, 2022 at 1:46 PM Aleh Linkin via curl-library <
>>>> curl-library_at_lists.haxx.se> wrote:
>>>>
>>>>> 1. easy_handles
>>>>> ```
>>>>> SET_OPT(CURLOPT_ERRORBUFFER, m_errorBuffer);
>>>>> setVerbose(true);
>>>>> setUrl(url);
>>>>> setTimeout(timeout);
>>>>> setConnectTimeout(connectTimeout);
>>>>> setProxy(proxy);
>>>>> setProxyPort(proxyPort);
>>>>> setHttpHeaders(httpHeaders);
>>>>> SET_OPT(CURLOPT_WRITEFUNCTION, writeFunction);
>>>>> SET_OPT(CURLOPT_WRITEDATA, m_requestState);
>>>>> SET_OPT(CURLOPT_DEBUGFUNCTION, CurlRequest::debugFunction);
>>>>> SET_OPT(CURLOPT_DEBUGDATA, this);
>>>>> SET_OPT(CURLOPT_PRIVATE, this);
>>>>> SET_OPT(CURLOPT_NOSIGNAL, 1L);
>>>>> ```
>>>>>
>>>>> 2. HTTPS. TLS based on NSS SSL backend
>>>>>
>>>>> 3. Yes. I've used perf and valgrind-cachgrind but I can't take the
>>>>> reason from results. For example perf part:
>>>>> ```
>>>>> Samples: 1K of event 'cycles:uppp', Event count (approx.): 9397031894
>>>>> Children Self Command Shared Object Symbol
>>>>> - 23.25% 2.43% lecappqry.1 [unknown] [.]
>>>>> 0000000000000000
>>>>> - 21.58% 0
>>>>> 5.29% _int_malloc
>>>>> 1.42% free
>>>>> 1.13% Curl_http
>>>>> 1.07% _int_free
>>>>> + 0.88% __clock_gettime
>>>>> 0.72% _IO_vsnprintf
>>>>> 0.71% __libc_calloc
>>>>> + 5.07% 0.00% lecappqry.1 [unknown] [k]
>>>>> 0x0000000000667849
>>>>> + 5.07% 0.00% lecappqry.1 [unknown] [k]
>>>>> 0x00007f33507ad900
>>>>> + 5.07% 4.87% lecappqry.1 lecappqry_grpc [.]
>>>>> dprintf_formatf
>>>>> + 4.88% 4.71% lecappqry.1 libc-2.17.so [.] _int_free
>>>>> ```
>>>>>
>>>>> On Mon, May 30, 2022 at 9:07 PM Patrick Schlangen <
>>>>> patrick_at_schlangen.me> wrote:
>>>>>
>>>>>> Am 30.05.2022 um 16:23 schrieb Aleh Linkin via curl-library <
>>>>>> curl-library_at_lists.haxx.se>:
>>>>>> >
>>>>>> > I understand. Thank you.
>>>>>> > But why multi_socket_action eats about twice time more CPU then
>>>>>> curl_multi_perform called in cycle with 10ms pause?
>>>>>>
>>>>>> Can you supply some more details about how the easy handle is
>>>>>> configured and the type of transfers you are running? Is it HTTP? Are you
>>>>>> using TLS? Did you use perf or a similar profiler yet to see where time is
>>>>>> spent?
>>>>>>
>>>>>> Thanks,
>>>>>>
>>>>>> Patrick
>>>>>>
>>>>>> --
>>>>> Unsubscribe: https://lists.haxx.se/listinfo/curl-library
>>>>> Etiquette: https://curl.haxx.se/mail/etiquette.html
>>>>>
>>>> --
>> Unsubscribe: https://lists.haxx.se/listinfo/curl-library
>> Etiquette: https://curl.haxx.se/mail/etiquette.html
>>
>
-- Unsubscribe: https://lists.haxx.se/listinfo/curl-library Etiquette: https://curl.haxx.se/mail/etiquette.htmlReceived on 2022-06-01