Curl upgrade from 7.65.1 to 7.70.0 causing http response processing performance degradation

Hello All,

I hope everyone is well. We consume the CURL library in our product
(running on Windows) and recently did a version upgrade from 7.65.1 to
7.70.0. There has been some noticeable performance degradation since this
upgrade.

*Scenario*
We are connecting to Google Big Query and trying to download data around
13mb. Download time has gone up from 1.8 secs (CURL version 7.65.1) to 6.2
sec (7.70.0). This slowness amplifies as the size of http response
increases or number of requests go up.

Upon investigating the issue, it has come to attention that we use PAUSE
and UNPAUSE features. We maintain a queue that can hold 20 items of size
16KB. Receive data callback from the library triggers injecting data into
the queue and upon reaching max queue capacity we pause the connection. As
soon as we dequeue the items, connection is unpaused and data continues to
flow. Before the library upgrade this flow performed well.

We did a quick experiment of increasing queue size to 5000 and it led to
download time of 1.8 secs which is the same as pre library upgrade time.
Based on this observation, it seems pausing and unpausing causes some kind
of slowness in case of 7.70.0., of course this is an hypothesis.

Question - How has pause/unpause behavior changed in 7.70.0? Does anyone
have insight what might be happening here?

Thanks for taking time to read this query.

Best,
Kunal

-----------------------------------------------------------
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-users
Etiquette: https://curl.haxx.se/mail/etiquette.html
Received on 2020-09-08