curl / Mailing Lists / curl-library / Single Mail
Buy commercial curl support from WolfSSL. We help you work out your issues, debug your libcurl applications, use the API, port to new platforms, add new features and more. With a team lead by the curl founder himself.

Re: Memory bloat in libcurl when using http/2

From: Daniel Stenberg via curl-library <>
Date: Mon, 3 Feb 2020 15:24:44 +0100 (CET)

On Fri, 31 Jan 2020, Sangamkar, Dheeraj via curl-library wrote:

> Environment: Debian 10 with libcurl 7.64.0-4. (some minor patches), nghttp2
> version 1.36.

> Problem: Memory consumption of application keeps growing when read a large
> amount of data using libcurl over http/2 over tls. It is released later or
> during application shutdow, so, I donĘt think there is a leak.

I believe the key to this problem you're seeing is the pausing of a HTTP/2
transfer/stream. Looking briefly into this, I don't think we currently handle
pausing HTTP/2 streams properly - in particular when there are other streams
over the same HTTP/2 connections that are being read.

Pausing a HTTP/2 stream *should* zero the flow control window to prevent the
other side to continue sending data until the stream is unpaused again. I
wonder if our current flow control approach might also contribute...

> Does libcurlĘs interfacing with nghttp2 provide a way for a slow reader to
> push back to the server during GET?

HTTP/2 doesn't really "push back" but it advertises a "window" for how much
the peer is allowed to send and that window should be adjusted accordingly.

I suspect we have work to do here.

> How can I fix this? What files and functions should I be messing with in
> libcurl code to tell the http server via nghttp2 that I donĘt want to
> receive any more data for a GET response body beyond a certain limit, say
> 1MB per GET.

If you don't want to receive more, why not just return an error from the write
callback when you're at that point and that tranfer ends?

> (Note: This is similar to a problem fixed last year for http/1.1 GETs in
> libcurl)

I don't think it is. The symptoms might be similar though.

  / | Commercial curl support up to 24x7 is available!
                   | Private help, bug fixes, support, ports, new features

Received on 2020-02-03