Re: Slow performance using multi_curl_* with HTTPs requests

From: Jess Sewovon <jesssewovon_at_gmail.com>
Date: Wed, 23 Jun 2021 09:15:43 +0000

Have you done the tests via command line without php? Can I see your php
file content?
I give you these 2 links, maybe can help you:
http://www.onlineaspect.com/2009/01/26/how-to-use-curl_multi-without-blocking/
https://webkul.com/blog/simultaneous-curl-requests-in-php/

Le mer. 23 juin 2021 à 08:46, Thomas Stähle <staelche_at_gmail.com> a écrit :

> Hi,
>
> I am experiencing performance issues using multi curl combined with HTTPs
> requests on a lately updated OS. It seems only related to multi curl and
> HTTPs. Single, serialized, curl requests with HTTPs do not show any slow
> down. So it seems not to be related to e.g. the OpenSSL version or the
> SSL/TLS Handshake. I am not quite sure if this is related to the PHP side
> of the multi curl implementation or libcurl multi curl interface...
>
> I ran some tests to track the problem down but until now without any
> success.
>
> The tests I did do either 100 requests in a row (no multi curl; script
> name curl_plain.php) or 100 times 10 requests in parallel with multi curl
> (curl_multi.php). Both call a PHP script just sleeping for 1 second. One
> time I called it via HTTP the other time via HTTPs. I ran all the tests on
> both nodes. One node is on Debian Jessie (the old node). The other one is
> on Debian Stretch (the new node). The following run times are all above 100
> seconds since it can't get faster with a sleep of 1 second called 100
> times. The interesting part is the overhead above the 100 seconds.
>
> The slow down happened with the upgrade from Jessie to Stretch. I tested
> it with Buster and with PHP7.4, as well. All combinations show the same
> issue. So it does not go away with any further upgrade.
>
> Since I am heavily relying on a performant multi_curl for some processes
> it is necessary for me to resolve the performance issue.
>
> The old node:
> Debian Jessie (8.6)
> PHP 7.1.17-1+0~20180505045956.17+jessie~1.gbpde69c6
> OpenSSL 1.0.1t
> libcurl(3) 7.38.0-4+deb8u4
>
> The new node:
> Debian Stretch (9.13)
> PHP 7.1.33-37+0~20210501.55+debian9~1.gbpcb9127
> OpenSSL 1.1.0l
> libcurl(3) 7.52.1-5+deb9u13
>
> Both nodes are exactly the same hardware wise. It is a dual CPU system
> with NUMA activated to share the memory between both sockets. Both nodes
> are located in the same network and call, of course, the same script on
> another node in a different network.
>
> The tests with HTTP:
> old $ php curl_plain.php
> Total: 101.47104287148 seconds
> old $ php curl_multi.php
> Total: 101.5175549984 seconds
> new $ php curl_plain.php
> Total: 101.44491195679 seconds
> new $ php curl_multi.php
> Total: 101.5433011055 seconds
> -> no issue
>
> The tests with HTTPs:
> old$ php src/curl_plain.php
> Total: 103.15611720085 seconds
> old$ php src/curl_multi.php
> Total: 104.17670083046 seconds
> new$ php src/curl_plain.php
> Total: 103.91886115074 seconds
> new$ php src/curl_multi.php
> Total: 110.56924200058 seconds
> -> It makes sense that with multi_curl there is a little bit more overhead
> but the big gap on the new node is a little bit too much I would say.
>
> Does anyone have an idea what the issue could be?
>
> Kind regards
> Thomas
> _______________________________________________
> https://cool.haxx.se/cgi-bin/mailman/listinfo/curl-and-php
>


_______________________________________________
https://cool.haxx.se/cgi-bin/mailman/listinfo/curl-and-php
Received on 2021-06-23