cURL / Mailing Lists / curl-library / Single Mail

curl-library

Re: Massive HTTP/2 parallel requests

From: Molina <jose.molina_at_cern.ch>
Date: Mon, 14 Mar 2016 19:02:31 +0100

> That's a really excessive buffer size. from 32K to 16MB - and this is per HTTP/2 connection. Can you also present what your test results are with some buffer sizes in between?
You are totally right. Using the default one (32KB) gives me the same performance, so it was obviously a mistake to increase its value. I updated the changes in my repository [1].

Concerning different window sizes, just as an orientation in order to find a “sufficiently large” value, here I have some tests:
16MB: 0m32s
8MB: 0m33s
4MB: 0m39s
2MB: 0m48s
1MB: 1m35s

> Additionally, there should be no need to set window size and buffer size to the same values. I question the need for setting the buffer size so large but I can understand why you need a fairly large window size. (I've already done HTTP/2 downloads with curl way faster than the browsers do on my computer with that 32K buffer size.)
>
> Can you elaborate on why you need to set NGHTTP2_SETTINGS_MAX_CONCURRENT_STREAMS and why to 64?
No special reason, I simply wanted to emulate exactly the same conditions my initial test with nghttp2 had. On a related matter, the server might indicate via SETTINGS that he’s not able to handle the default value CURL has at the moment, which is the same default nghttp2 has, i.e. 100 parallel streams. If that happens, should CURL decrease (or increase!) its own NGHTTP2_SETTINGS_MAX_CONCURRENT_STREAMS to the minimum of both client and server?

> So it sounds like the overall throughput is governed by half the maximum window size. I like the sound of an auto scaling window, you could PING (frame) the server early in the connection and use that as an early indicator of RTT and scale up window size in some manner. Or you could be more aggressive with sending WINDOW_UPDATES e.g. after every frame received.
Yes! That would be the most efficient way, I think. It would also be the most complicated to implement, though. On the other hand, sending many WINDOW_UPDATES would imply more traffic from the client to the server, and that might be a potencial inconvenient for many of the users, who usually have a pretty poor upload speed and a decent download speed due to the asymmetric DSL, isn’t it?

Cheers,
José

[1] https://github.com/curl/curl/compare/master...Moliholy:http2-window <https://github.com/curl/curl/compare/master...Moliholy:http2-window>

-------------------------------------------------------------------
List admin: https://cool.haxx.se/list/listinfo/curl-library
Etiquette: https://curl.haxx.se/mail/etiquette.html
Received on 2016-03-14