Buy commercial curl support. We
help you work out your issues, debug your libcurl applications, use the API,
port to new platforms, add new features and more. With a team lead by the
curl founder Daniel himself.
Re: Increase in CPU usage in 8.7.1 vs 8.6.0 for rate-limited downloads
- Contemporary messages sorted: [ by date ] [ by thread ] [ by subject ] [ by author ] [ by messages with attachments ]
From: Dmitry Karpov via curl-library <curl-library_at_lists.haxx.se>
Date: Wed, 15 May 2024 19:26:57 +0000
I just want to refresh that this commit helped to restore the precision of rate throttling.
Before it, the speed deviations from the limit were more than 10% in certain cases.
And depending on the network speed, they occurred on relatively big downloads (i.e. 5 MB) as well.
I think that CPU usage with this commit should be compared to previous releases, like 7.84.0, which had a quite precise speed limit.
Thanks,
Dmitry Karpov
From: curl-library <curl-library-bounces_at_lists.haxx.se> On Behalf Of David Pfitzner via curl-library
Sent: Wednesday, May 15, 2024 4:04 AM
To: Daniel Stenberg <daniel_at_haxx.se>
Cc: David Pfitzner <dpfitzner_at_netflix.com>; David Pfitzner via curl-library <curl-library_at_lists.haxx.se>
Subject: [EXTERNAL] Re: Increase in CPU usage in 8.7.1 vs 8.6.0 for rate-limited downloads
On Wed, May 15, 2024 at 5:31 PM Daniel Stenberg <daniel_at_haxx.se<mailto:daniel_at_haxx.se>> wrote:
On Wed, 15 May 2024, David Pfitzner via curl-library wrote:
> From testing, this change appears to be due to this commit:
>
> https://github.com/curl/curl/commit/db5c9f4f9e0779b49624752b135281a0717b277b
So just reverting this single commit makes it "good" again?
Correct.
> Perhaps it would be useful for a user of libcurl to be able to (somehow)
> control this tradeoff between rate-limiting accuracy and CPU usage?
Perhaps getting more data would be a first step. How big difference in
rate-limit accuracy does this commit make in your case?
I have not looked at that carefully, but casually I don't see much difference if any. But possibly closer inspection may find a systematic difference. I also suspect one may see bigger differences in some regime different to what I'm looking at - eg, smaller files.
-- David
Date: Wed, 15 May 2024 19:26:57 +0000
I just want to refresh that this commit helped to restore the precision of rate throttling.
Before it, the speed deviations from the limit were more than 10% in certain cases.
And depending on the network speed, they occurred on relatively big downloads (i.e. 5 MB) as well.
I think that CPU usage with this commit should be compared to previous releases, like 7.84.0, which had a quite precise speed limit.
Thanks,
Dmitry Karpov
From: curl-library <curl-library-bounces_at_lists.haxx.se> On Behalf Of David Pfitzner via curl-library
Sent: Wednesday, May 15, 2024 4:04 AM
To: Daniel Stenberg <daniel_at_haxx.se>
Cc: David Pfitzner <dpfitzner_at_netflix.com>; David Pfitzner via curl-library <curl-library_at_lists.haxx.se>
Subject: [EXTERNAL] Re: Increase in CPU usage in 8.7.1 vs 8.6.0 for rate-limited downloads
On Wed, May 15, 2024 at 5:31 PM Daniel Stenberg <daniel_at_haxx.se<mailto:daniel_at_haxx.se>> wrote:
On Wed, 15 May 2024, David Pfitzner via curl-library wrote:
> From testing, this change appears to be due to this commit:
>
> https://github.com/curl/curl/commit/db5c9f4f9e0779b49624752b135281a0717b277b
So just reverting this single commit makes it "good" again?
Correct.
> Perhaps it would be useful for a user of libcurl to be able to (somehow)
> control this tradeoff between rate-limiting accuracy and CPU usage?
Perhaps getting more data would be a first step. How big difference in
rate-limit accuracy does this commit make in your case?
I have not looked at that carefully, but casually I don't see much difference if any. But possibly closer inspection may find a systematic difference. I also suspect one may see bigger differences in some regime different to what I'm looking at - eg, smaller files.
-- David
-- Unsubscribe: https://lists.haxx.se/mailman/listinfo/curl-library Etiquette: https://curl.se/mail/etiquette.htmlReceived on 2024-05-15