Re: Memory bloat in libcurl when using http/2
Date: Mon, 3 Feb 2020 15:24:44 +0100 (CET)
On Fri, 31 Jan 2020, Sangamkar, Dheeraj via curl-library wrote:
> Environment: Debian 10 with libcurl 7.64.0-4. (some minor patches), nghttp2
> version 1.36.
> Problem: Memory consumption of application keeps growing when read a large
> amount of data using libcurl over http/2 over tls. It is released later or
> during application shutdow, so, I donĘt think there is a leak.
I believe the key to this problem you're seeing is the pausing of a HTTP/2
transfer/stream. Looking briefly into this, I don't think we currently handle
pausing HTTP/2 streams properly - in particular when there are other streams
over the same HTTP/2 connections that are being read.
Pausing a HTTP/2 stream *should* zero the flow control window to prevent the
other side to continue sending data until the stream is unpaused again. I
wonder if our current flow control approach might also contribute...
> Does libcurlĘs interfacing with nghttp2 provide a way for a slow reader to
> push back to the server during GET?
HTTP/2 doesn't really "push back" but it advertises a "window" for how much
the peer is allowed to send and that window should be adjusted accordingly.
I suspect we have work to do here.
> How can I fix this? What files and functions should I be messing with in
> libcurl code to tell the http server via nghttp2 that I donĘt want to
> receive any more data for a GET response body beyond a certain limit, say
> 1MB per GET.
If you don't want to receive more, why not just return an error from the write
callback when you're at that point and that tranfer ends?
> (Note: This is similar to a problem fixed last year for http/1.1 GETs in
I don't think it is. The symptoms might be similar though.
-- / daniel.haxx.se | Commercial curl support up to 24x7 is available! | Private help, bug fixes, support, ports, new features | https://www.wolfssl.com/contact/
Received on 2020-02-03