Re: curl-multi-socket interface too slow

From: James Read via curl-library <curl-library_at_lists.haxx.se>
Date: Wed, 1 Jun 2022 18:42:31 +0100

On Wed, Jun 1, 2022 at 2:22 PM Aleh Linkin <aleh.linkin_at_idt.net> wrote:

> Thanks for the update. Could you be a bit more specific? What do you mean
> under "problem with the TCP/IP stack"? If I'd like to use
> curl_multi_socket_action I had to sacrifice some CPU?
>

Some time ago I posted a question related on stackoverflow:
https://stackoverflow.com/questions/70584121/why-doesnt-my-epoll-based-program-improve-performance-by-increasing-the-number


The question includes no cURL based code which demonstrates the limitations
of epoll with the current TCP/IP stack. I don't know exactly what the
problem with the stack is. I'm still working on it.

James Read


>
> On Wed, Jun 1, 2022 at 4:04 PM James Read <jamesread5737_at_gmail.com> wrote:
>
>> The problem is not with cURL. I have also run into these problems using
>> cURL. I recoded using sockets and epoll and got the same results. The
>> problem is with the TCP/IP stack.
>>
>> On Wed, Jun 1, 2022 at 1:46 PM Aleh Linkin via curl-library <
>> curl-library_at_lists.haxx.se> wrote:
>>
>>> 1. easy_handles
>>> ```
>>> SET_OPT(CURLOPT_ERRORBUFFER, m_errorBuffer);
>>> setVerbose(true);
>>> setUrl(url);
>>> setTimeout(timeout);
>>> setConnectTimeout(connectTimeout);
>>> setProxy(proxy);
>>> setProxyPort(proxyPort);
>>> setHttpHeaders(httpHeaders);
>>> SET_OPT(CURLOPT_WRITEFUNCTION, writeFunction);
>>> SET_OPT(CURLOPT_WRITEDATA, m_requestState);
>>> SET_OPT(CURLOPT_DEBUGFUNCTION, CurlRequest::debugFunction);
>>> SET_OPT(CURLOPT_DEBUGDATA, this);
>>> SET_OPT(CURLOPT_PRIVATE, this);
>>> SET_OPT(CURLOPT_NOSIGNAL, 1L);
>>> ```
>>>
>>> 2. HTTPS. TLS based on NSS SSL backend
>>>
>>> 3. Yes. I've used perf and valgrind-cachgrind but I can't take the
>>> reason from results. For example perf part:
>>> ```
>>> Samples: 1K of event 'cycles:uppp', Event count (approx.): 9397031894
>>> Children Self Command Shared Object Symbol
>>> - 23.25% 2.43% lecappqry.1 [unknown] [.]
>>> 0000000000000000
>>> - 21.58% 0
>>> 5.29% _int_malloc
>>> 1.42% free
>>> 1.13% Curl_http
>>> 1.07% _int_free
>>> + 0.88% __clock_gettime
>>> 0.72% _IO_vsnprintf
>>> 0.71% __libc_calloc
>>> + 5.07% 0.00% lecappqry.1 [unknown] [k]
>>> 0x0000000000667849
>>> + 5.07% 0.00% lecappqry.1 [unknown] [k]
>>> 0x00007f33507ad900
>>> + 5.07% 4.87% lecappqry.1 lecappqry_grpc [.]
>>> dprintf_formatf
>>> + 4.88% 4.71% lecappqry.1 libc-2.17.so [.] _int_free
>>> ```
>>>
>>> On Mon, May 30, 2022 at 9:07 PM Patrick Schlangen <patrick_at_schlangen.me>
>>> wrote:
>>>
>>>> Am 30.05.2022 um 16:23 schrieb Aleh Linkin via curl-library <
>>>> curl-library_at_lists.haxx.se>:
>>>> >
>>>> > I understand. Thank you.
>>>> > But why multi_socket_action eats about twice time more CPU then
>>>> curl_multi_perform called in cycle with 10ms pause?
>>>>
>>>> Can you supply some more details about how the easy handle is
>>>> configured and the type of transfers you are running? Is it HTTP? Are you
>>>> using TLS? Did you use perf or a similar profiler yet to see where time is
>>>> spent?
>>>>
>>>> Thanks,
>>>>
>>>> Patrick
>>>>
>>>> --
>>> Unsubscribe: https://lists.haxx.se/listinfo/curl-library
>>> Etiquette: https://curl.haxx.se/mail/etiquette.html
>>>
>>

-- 
Unsubscribe: https://lists.haxx.se/listinfo/curl-library
Etiquette:   https://curl.haxx.se/mail/etiquette.html

Received on 2022-06-01