Re: Curl upgrade from 7.65.1 to 7.70.0 causing http response processing performance degradation

Hi Daniel,

>
>> I would suggest three action items going forward:
>>
>> - Try 7.72.0 and see if the issue is still there
>> - If it is, consider bisecting to find exactly which commit that
>> introduced
>> the problem (as it'll certainly help us to understand what's going on)
>> - If bisecting doesn't work or whatever, try writing up an example
>> application that reproduces the problem (basically using the same
>> logic
>> your application does) so that we can help out debugging it from our
>> side.
>>
>> If all that fails, then debugging this yourself might be the only way...
>
> Thanks Daniel for the prompt response. I will try the steps you have
> recommended and respond back with outcomes.
>

I further investigated the issue. As I mentioned earlier, we use the
connection pause/unpause feature. If the connection is paused then the
library uses a temporary buffer (pausewrite function
<https://github.com/curl/curl/search?q=pausewrite&unscoped_q=pausewrite>)
for holding the data. I noticed that this buffer continues to increase
(buffer size reaches upto 7mb for 13mb response) and malloc, memcpy calls
on the buffer of this size causes slowness. Further investigation revealed
that buffer size increase has been taking place since the following change
<https://github.com/curl/curl/commit/e040146f22608fd92c44be2447a6505141a8a867>
was introduced in the library. Before this change, we did not notice any
performance degradation. I compared buffer size before and after the
change, screenshot attached here. How do you think we can proceed from here?
[image: image.png]

https://github.com/curl/curl/commit/e040146f22608fd92c44be2447a6505141a8a867

>

-----------------------------------------------------------
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-users
Etiquette: https://curl.haxx.se/mail/etiquette.html