curl / Mailing Lists / curl-library / Single Mail

curl-library

Re: [curl/curl] 04b61f: progress: fix rate-limit for high speed transfers

From: Ben Greear <greearb_at_candelatech.com>
Date: Tue, 20 Mar 2018 16:22:49 -0700

On 03/20/2018 04:14 PM, Daniel Stenberg wrote:
> On Tue, 20 Mar 2018, Ben Greear wrote:
>
>>> This new logic updates the rate limit "window" to be no shorter than the
>>> last three seconds and only updating the timestamps for this when
>>> switching between the states TOOFAST/PERFORM.
>>
>> As far as I can tell, this patch is broken. I am downloading a 20MB file using ftp, with a program compiled against lib-curl. With this patch, download
>> spikes to 150Mbps (or probably higher, depending on resolution), and then has a long pause for the next URL (that pause might be caused by my own rate-ctrl
>> across URLs).
>
> Yeah, this fix basically reversed the problem and https://github.com/curl/curl/issues/2406 was subsequently filed, although phrased more like a discusssion.
>
> I think I need to backpedal a bit and make the period to check be something like 500 milliseconds or something. At least not longer than a second I think.
>
>> And/or, what speed showed the previous problem you are trying to fix?
>
> The previous problem was that for high speed transfers on for example localhost, it could do the second check the same millisecond as the previous which made
> the delta time to be zero, thus not limiting it at all.
>
> Zero was wrong and 3000 milliseconds is wrong in the other direction...

So, how about 1ms? That will likely be precise enough, and you should be able to
do the math such that rounding errors are not too bad. 500ms is a very long time
in the realm of 1 and 10Gbps networks....

I didn't actually look at the math in your rate-limiting code, but I've been using 1ms
time intervals for such stuff for years and it works well for me...

Thanks,
Ben

-- 
Ben Greear <greearb_at_candelatech.com>
Candela Technologies Inc  http://www.candelatech.com
-------------------------------------------------------------------
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library
Etiquette:   https://curl.haxx.se/mail/etiquette.html
Received on 2018-03-21