Re: Curl upgrade from 7.65.1 to 7.70.0 causing performance degradation

On Mon, 14 Sep 2020, Kunal Chandarana via curl-library wrote:

> I further investigated the issue. As I mentioned earlier, we use the
> connection pause/unpause feature. If the connection is paused then the
> library uses a temporary buffer (pausewrite function
> <https://github.com/curl/curl/search?q=pausewrite&unscoped_q=pausewrite>)
> for holding the data. I noticed that this buffer continues to increase
> (buffer size reaches upto 7mb for 13mb response) and malloc, memcpy calls on
> the buffer of this size causes slowness. Further investigation revealed that
> buffer size increase has been taking place since the following change
> <https://github.com/curl/curl/commit/e040146f22608fd92c44be2447a6505141a8a867>
> was introduced in the library. Before this change, we did not notice any
> performance degradation. I compared buffer size before and after the change,
> screenshot attached here. How do you think we can proceed from here for
> fixing this issue?

The need for a "pause buffer" is unfortunate to begin with. There are however
times (when using HTTP/2 primarily) when curl can't stop the transfer
immediately and thus it needs somewhere to store the incoming data that
arrives after the user told libcurl to pause.

The size of that dynamic buffer then simply depends on how much data that
arrives while the connection is paused. To me, it seems unlikely that the
commit mentioned above is able to add to that data amount. In fact, it should
help drain the existing buffer bettere, since that change is in the *unpause*
logic.

-- 
  / daniel.haxx.se | Commercial curl support up to 24x7 is available!
                   | Private help, bug fixes, support, ports, new features
                   | https://www.wolfssl.com/contact/
-------------------------------------------------------------------
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library
Etiquette:   https://curl.haxx.se/mail/etiquette.html