cURL / Mailing Lists / curl-library / Single Mail

curl-library

Re: Massive HTTP/2 parallel requests

From: Daniel Stenberg <daniel_at_haxx.se>
Date: Fri, 11 Mar 2016 23:56:21 +0100 (CET)

On Fri, 11 Mar 2016, Molina wrote:

> To be honest, I am not using any browser in my benchmarks, mainly because
> (as far as I understood) they only support HTTP/2 over SSL, which is not the
> case of the product I’m working for, neither the case of the server I’m
> using. Also, I haven’t found any other client that allows the user to
> specify the per-connection and per-stream window size, which is also
> mentioned in this email chain.

No, but browsers are meant to be used even on high bandwidth high latency
connections so they want to do h2 transfers with high performance in this
situation as well so I was just curious if they actually do. If they do, we
could learn from their tricks. If they don't, we could submit bug reports and
fix them (too).

>> In general though, libcurl doesn't really make much use of the HTTP/2 flow
>> control features so it would probably make sense to more or less by default
>> use a pretty large window.

> Firstly, a pretty large window that is not efficiently used means a waste of
> memory.

Not necessarily for all windows. A HTTP/2 stream window is just a promise that
we will accept that much data if the other end sends it. In libcurl's case,
promising a bigger HTTP/2 window wouldn't change memory consumption with a
single byte.

The TCP receive buffer options adds memory usage but that's just one buffer
for the entire connection, and it isn't necessarily so that we need to
increase that window always just because we might need a bigger HTTP/2 window.

We need to promise a big enough HTTP/2 window that allows the other end to
send at full speed and for us to send back window updates in due time. I
wonder if we could make so that we get some RTT timing back from nghttp2 and
we could use that as input for how large window to set... Or perhaps we could
even make such a window scaling option into nghttp2 itself.

> It doesn’t mean a user has to every time specify that option, and it’s not
> incompatible with having a sensible default. It’s just that providing a
> --window-bits=<N> and --connection-window-bits=<N>”-like options in CURL
> would be an possibility to consider at least for HTTP/2.

Sure, and I agree that if we have to add such window size options to get good
transfer speeds in these situations we should of course do exactly that. I
just think that exploring handling the window problem without any options
would be an even better change so I think it is worth discussing and exploring
problem before we add window sizing options.

-- 
  / daniel.haxx.se

-------------------------------------------------------------------
List admin: https://cool.haxx.se/list/listinfo/curl-library
Etiquette: https://curl.haxx.se/mail/etiquette.html
Received on 2016-03-11