Buy commercial curl support. We
help you work out your issues, debug your libcurl applications, use the API,
port to new platforms, add new features and more. With a team lead by the
curl founder Daniel himself.
Re: CPU usage since 8.7.1
- Contemporary messages sorted: [ by date ] [ by thread ] [ by subject ] [ by author ] [ by messages with attachments ]
From: Stefan Eissing via curl-library <curl-library_at_lists.haxx.se>
Date: Thu, 30 Oct 2025 10:52:56 +0100
> Am 30.10.2025 um 02:47 schrieb Gleb Smirnoff <glebius_at_glebi.us>:
>
> On Wed, Oct 29, 2025 at 09:27:35AM +0100, Stefan Eissing wrote:
> S> > On Tue, Oct 28, 2025 at 11:06:05PM +0100, Daniel Stenberg wrote:
> S> > D> > This may actually help our case!
> S> > D> > I'm going to give a try to unmodified 8.16.0.
> ...
> S> > The regression was discovered by a huge A/B test of a system overall,
> S> > that involves many machines, many hours and a ton of traffic. What
> S> > I'm going to do now is to re-run this A/B test with 8.16.0, to avoid
> S> > relying on year old data.
> S>
> S> Waiting for your results then. The "limit-rate revisisted" PR is https://github.com/curl/curl/pull/18454
> S>
> S> I noticed "busy" loops where we expired transfers that are on a speed limits. Hope that addresses the cpu issue for you. As to the accuracy, we'll see what you find in your setup.
>
> We had run our large test and I can confirm that 8.16.0 doesn't need our
> patch any more!
>
> I guess the chunk to transfer.c in 24badd29f is the key:
>
> --- a/lib/transfer.c
> +++ b/lib/transfer.c
> _at__at_ -280,9 +280,10 _at__at_ static CURLcode sendrecv_dl(struct Curl_easy *data,
>
> if(bytestoread && data->set.max_recv_speed > 0) {
> /* In case of speed limit on receiving: if this loop already got
> - * data, break out. If not, limit the amount of bytes to receive.
> - * The overall, timed, speed limiting is done in multi.c */
> - if(total_received)
> + * a quarter of the quota, break out. We want to stutter a bit
> + * to keep in the limit, but too small receives will just cost
> + * cpu unnecessarily. */
> + if(total_received >= (data->set.max_recv_speed / 4))
> break;
> if(data->set.max_recv_speed < (curl_off_t)bytestoread)
> bytestoread = (size_t)data->set.max_recv_speed;
>
> Thanks!
Happy to hear that I can watch Netflix with less cost for you soon!
=)
Cheers,
Stefan
>
> --
> Gleb Smirnoff
Date: Thu, 30 Oct 2025 10:52:56 +0100
> Am 30.10.2025 um 02:47 schrieb Gleb Smirnoff <glebius_at_glebi.us>:
>
> On Wed, Oct 29, 2025 at 09:27:35AM +0100, Stefan Eissing wrote:
> S> > On Tue, Oct 28, 2025 at 11:06:05PM +0100, Daniel Stenberg wrote:
> S> > D> > This may actually help our case!
> S> > D> > I'm going to give a try to unmodified 8.16.0.
> ...
> S> > The regression was discovered by a huge A/B test of a system overall,
> S> > that involves many machines, many hours and a ton of traffic. What
> S> > I'm going to do now is to re-run this A/B test with 8.16.0, to avoid
> S> > relying on year old data.
> S>
> S> Waiting for your results then. The "limit-rate revisisted" PR is https://github.com/curl/curl/pull/18454
> S>
> S> I noticed "busy" loops where we expired transfers that are on a speed limits. Hope that addresses the cpu issue for you. As to the accuracy, we'll see what you find in your setup.
>
> We had run our large test and I can confirm that 8.16.0 doesn't need our
> patch any more!
>
> I guess the chunk to transfer.c in 24badd29f is the key:
>
> --- a/lib/transfer.c
> +++ b/lib/transfer.c
> _at__at_ -280,9 +280,10 _at__at_ static CURLcode sendrecv_dl(struct Curl_easy *data,
>
> if(bytestoread && data->set.max_recv_speed > 0) {
> /* In case of speed limit on receiving: if this loop already got
> - * data, break out. If not, limit the amount of bytes to receive.
> - * The overall, timed, speed limiting is done in multi.c */
> - if(total_received)
> + * a quarter of the quota, break out. We want to stutter a bit
> + * to keep in the limit, but too small receives will just cost
> + * cpu unnecessarily. */
> + if(total_received >= (data->set.max_recv_speed / 4))
> break;
> if(data->set.max_recv_speed < (curl_off_t)bytestoread)
> bytestoread = (size_t)data->set.max_recv_speed;
>
> Thanks!
Happy to hear that I can watch Netflix with less cost for you soon!
=)
Cheers,
Stefan
>
> --
> Gleb Smirnoff
-- Unsubscribe: https://lists.haxx.se/mailman/listinfo/curl-library Etiquette: https://curl.se/mail/etiquette.htmlReceived on 2025-10-30