Buy commercial curl support from WolfSSL. We help you work
out your issues, debug your libcurl applications, use the API, port to new
platforms, add new features and more. With a team lead by the curl founder
himself.
Re: Large download speed deviations for CURLOPT_MAX_RECV_SPEED_LARGE in 8.4.0
- Contemporary messages sorted: [ by date ] [ by thread ] [ by subject ] [ by author ] [ by messages with attachments ]
From: Daniel Stenberg via curl-library <curl-library_at_lists.haxx.se>
Date: Sat, 2 Dec 2023 17:57:33 +0100 (CET)
On Sat, 2 Dec 2023, Dmitry Karpov via curl-library wrote:
> No, buffering is the same. I explicitly set 1MB socket receive buffer to
> avoid any default buffering differences between 8.4. and 7.84.
>
> The deviations are not sensitive to file sizes. They slightly vary between
> file sizes, but still are quite large for all file sizes.
I'm sure there are lots of factors around, as the rate limiting logic in curl
is very simple. To help the rate limiter somewhat, you can adjust the buffer
size accordingly and make sure it is not made larger than number of
bytes/second you want to transfer.
I don't think it helps that you post more numbers of your results. I want to
know if the math is right or wrong for you. The patch I posted would show
that.
Currently the rate limit stores a time stamp and the amount of data every
three seconds and then base the calculations on that. It (the three second
interval) was picked fairly arbitrarily out of the air, so I bet we can
consider alternatives of solving this and I'm certainly open for suggestions.
Note that any transfer that is completed within say six seconds or so, is not
likely to every get very accurate rate limiting because of this.
Accurate rate limiting is... hard.
Date: Sat, 2 Dec 2023 17:57:33 +0100 (CET)
On Sat, 2 Dec 2023, Dmitry Karpov via curl-library wrote:
> No, buffering is the same. I explicitly set 1MB socket receive buffer to
> avoid any default buffering differences between 8.4. and 7.84.
>
> The deviations are not sensitive to file sizes. They slightly vary between
> file sizes, but still are quite large for all file sizes.
I'm sure there are lots of factors around, as the rate limiting logic in curl
is very simple. To help the rate limiter somewhat, you can adjust the buffer
size accordingly and make sure it is not made larger than number of
bytes/second you want to transfer.
I don't think it helps that you post more numbers of your results. I want to
know if the math is right or wrong for you. The patch I posted would show
that.
Currently the rate limit stores a time stamp and the amount of data every
three seconds and then base the calculations on that. It (the three second
interval) was picked fairly arbitrarily out of the air, so I bet we can
consider alternatives of solving this and I'm certainly open for suggestions.
Note that any transfer that is completed within say six seconds or so, is not
likely to every get very accurate rate limiting because of this.
Accurate rate limiting is... hard.
-- / daniel.haxx.se | Commercial curl support up to 24x7 is available! | Private help, bug fixes, support, ports, new features | https://curl.se/support.html -- Unsubscribe: https://lists.haxx.se/mailman/listinfo/curl-library Etiquette: https://curl.se/mail/etiquette.htmlReceived on 2023-12-02