curl / Mailing Lists / curl-library / Single Mail
Buy commercial curl support from WolfSSL. We help you work out your issues, debug your libcurl applications, use the API, port to new platforms, add new features and more. With a team lead by the curl founder himself.

Re: Curl upgrade from 7.65.1 to 7.70.0 causing performance degradation

From: Kunal Chandarana via curl-library <>
Date: Wed, 16 Sep 2020 17:16:45 -0700

> The need for a "pause buffer" is unfortunate to begin with. There are
> however
> times (when using HTTP/2 primarily) when curl can't stop the transfer
> immediately and thus it needs somewhere to store the incoming data that
> arrives after the user told libcurl to pause.

Thanks for explaining it, I was curious about the need to use a pause

> The size of that dynamic buffer then simply depends on how much data that
> arrives while the connection is paused. To me, it seems unlikely that the
> commit mentioned above is able to add to that data amount. In fact, it
> should
> help drain the existing buffer bettere, since that change is in the
> *unpause*
> logic.

That makes sense. However let me briefly explain how we are using
unpause/pause (maybe usage is not as anticipated). We use
curl_multi_perform in our implementation.
1) We have a callback for CURLOPT_WRITEFUNCTION that injects 16KB (or less)
of data received from the library into a queue (as one item).
2) If the queue size exceeds 20 after pushing data (that is 20 items), we
*pause* the connection.
3) Another thread is reading from the queue and each dequeue call results
in *unpause* the connection
4) After unpausing connection, write function callback again starts
receiving data. If the queue reaches it's capacity, we *pause* the
connection again.
5) Reading from the queue results in *unpausing* connection.
This is how pausing and unpausing continues until the response is
processed. In your opinion, do you see the usage flawed?

The commit I mentioned earlier sets CURL_CSELECT_IN in case an unpause call
occurs. It opens up read-write flow here
the buffer
continues to grow. Please feel free to correct me if my understanding is
Your earlier recommendation was bisecting for pinpointing the commit that
is causing this issue. I revert the commit locally and performance
degradation is eliminated. What further investigation from my side would
help here? Do let me know and I would be happy to make progress in that


Received on 2020-09-17