curl / Mailing Lists / curl-library / Single Mail

curl-library

Re: Using CURLOPT_LOW_SPEED_LIMIT to detect a stalled request

From: Rory McCarthy via curl-library <curl-library_at_cool.haxx.se>
Date: Fri, 5 Oct 2018 02:35:53 +0000 (UTC)

 Looking at the debug output, all the data transfers in less than a second at a transfer speed in the 100's MB/s. It then stalls completely while waiting for a response for 6-7 seconds before timing out. Subsequent retries are able to complete in a reasonable time, so being able to time out sooner is key.
Thanks for confirming that CURLOPT_LOW_SPEED_LIMIT is compared against the average transfer rate. Would you accept a PR against the docs for CURLOPT_LOW_SPEED_LIMIT explaining this? On Thursday, 4 October 2018, 03:48:38 GMT-7, Daniel Stenberg <daniel_at_haxx.se> wrote:
 
 On Wed, 3 Oct 2018, Rory McCarthy via curl-library wrote:

> I set something like the following for the request:-
>    curl_easy_setopt(curl, CURLOPT_LOW_SPEED_TIME, 1L);
>    curl_easy_setopt(curl, CURLOPT_LOW_SPEED_LIMIT, 1L);
>
> I also added some debug logging via CURLOPT_DEBUGFUNCTION and
> CURLOPT_PROGRESSFUNCTION.
>
> While this works, it takes longer than expected to timeout.
>
> Reading through the libcurl source code, I think this is because it's using
> an average for the current transfer rate which gets compared to the
> CURLOPT_LOW_SPEED_LIMIT.

This issue or question is not as easy as it may sound and you'll find that a
lot of people will have different opinions on this:

    What is your current transfer speed?

It is very common for the network calls in libcurl to be "bursty" so that one
call first can get 16K and then there's a short while when noting arrives and
then it gets another larger chunk of data. For most people and use cases, it
makes sense to smoothen out those temporary ups and downs and just consider
the network speed as an average during the N most recent seconds. That's what
curl does.

So therefore, the "current transfer speed" in this context is indeed the speed
on average for the last 5 seconds. To go below 1 bytes/sec from a "real
speed", it will thus take all those 5 seconds.

> Is there a way to detect a stall without the delay?

I think your definition of "stall" is very strict and will easily trigger
false positives. It is not uncommon even for fully working transfers to get
brief pauses where no data arrives.

You can still easily implement your own network stall detection logic using
the progress callback that works exactly the way like it!

-- 
  / daniel.haxx.se  

-------------------------------------------------------------------
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library
Etiquette: https://curl.haxx.se/mail/etiquette.html
Received on 2018-10-05