Download
Browse source Changelog
Documentation
Project   Bug Bounty   Help us   Known bugs   TODO Protocols   CA bundle   HTTP Cookies   HTTP/2   SSL Certs Releases   Security   Version numbers   Vulnerabilities curl tool   man page   Tutorial   HTTP scripting Who and Why
libcurl
ABI API Competitors Examples Features Mailing list Related libs Using libcurl Tutorial Testimonials
Get Help
curl-library curl-users Mailing lists Book: Everything curl Video presentations Report a bug Paid support
Development
Autobuilds Code Style Contribute Deprecate Internals Release Notes Release Procedure Roadmap Run Tests Security Process Specifications Test curl
News
Changelog Release table
curl / Mailing Lists / curl-library / Single Mail
Buy commercial curl support from WolfSSL. We help you work out your issues, debug your libcurl applications, use the API, port to new platforms, add new features and more. With a team lead by the curl founder himself.

Re: Memory bloat in libcurl when using http/2

From: Daniel Stenberg via curl-library <curl-library_at_cool.haxx.se>
Date: Mon, 3 Feb 2020 15:24:44 +0100 (CET)

On Fri, 31 Jan 2020, Sangamkar, Dheeraj via curl-library wrote:

> Environment: Debian 10 with libcurl 7.64.0-4. (some minor patches), nghttp2
> version 1.36.

> Problem: Memory consumption of application keeps growing when read a large
> amount of data using libcurl over http/2 over tls. It is released later or
> during application shutdow, so, I don¢t think there is a leak.

I believe the key to this problem you're seeing is the pausing of a HTTP/2
transfer/stream. Looking briefly into this, I don't think we currently handle
pausing HTTP/2 streams properly - in particular when there are other streams
over the same HTTP/2 connections that are being read.

Pausing a HTTP/2 stream *should* zero the flow control window to prevent the
other side to continue sending data until the stream is unpaused again. I
wonder if our current flow control approach might also contribute...

> Does libcurl¢s interfacing with nghttp2 provide a way for a slow reader to
> push back to the server during GET?

HTTP/2 doesn't really "push back" but it advertises a "window" for how much
the peer is allowed to send and that window should be adjusted accordingly.

I suspect we have work to do here.

> How can I fix this? What files and functions should I be messing with in
> libcurl code to tell the http server via nghttp2 that I don¢t want to
> receive any more data for a GET response body beyond a certain limit, say
> 1MB per GET.

If you don't want to receive more, why not just return an error from the write
callback when you're at that point and that tranfer ends?

> (Note: This is similar to a problem fixed last year for http/1.1 GETs in
> libcurl)

I don't think it is. The symptoms might be similar though. 

-- 
  / daniel.haxx.se | Commercial curl support up to 24x7 is available!
                   | Private help, bug fixes, support, ports, new features
                   | https://www.wolfssl.com/contact/

-------------------------------------------------------------------
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library
Etiquette: https://curl.haxx.se/mail/etiquette.html
Received on 2020-02-03