cURL / Mailing Lists / curl-library / Single Mail

curl-library

Re: Massive HTTP/2 parallel requests

From: Rainer Jung <rainer.jung_at_kippdata.de>
Date: Wed, 2 Mar 2016 16:59:40 +0100

Am 02.03.2016 um 15:30 schrieb Molina:
> The results of curl —version are:
>
> curl 7.47.1 (x86_64-apple-darwin15.3.0) libcurl/7.47.1 OpenSSL/1.0.2f zlib/1.2.5 nghttp2/1.7.0
> Protocols: dict file ftp ftps gopher http https imap imaps ldap ldaps pop3 pop3s rtsp smb smbs smtp smtps telnet tftp
> Features: IPv6 Largefile NTLM NTLM_WB SSL libz TLS-SRP HTTP2 UnixSockets
>
> Concerning the server, I’m running an apache with mod_http2 enabled and the following HTTP/2-related configuration:
>
> LoadModule http2_module modules/mod_http2.so
> <IfModule http2_module>
> Protocols h2c http/1.1
> LogLevel http2:info
> </IfModule>
>
> I performed this very morning another test adding the following line: H2MaxSessionStreams 1000000
> I was just to make sure that the number of streams is higher than the number of parallel downloads for my test (1024). I realised looking at the logs that all my requests fit in the pipe, but still files are downloaded in 64-stream blocks! If I try with normal HTPP/1.1 connections the total test is around 20 times faster than this way.

Does setting H2MaxWorkers on the Apache side help? Also note, that the
Apache module is still experimental and in rapid development. Stefan
publishes in trunk but also in https://github.com/icing/mod_h2. The
github one is more recent and gets reintegrated into Apache every now
and then. So if you suspect a bug in the module, please also try the
github variant.

The github version has the following info in the entries for version 1.1.0:

* H2SessionExtraFiles are now shared between all sessions in the same
process.
    It now works like this:
    H2MaxWorkers * H2SessionFiles is assumed to be the total number of file
    handles that can be safely used for HTTP/2 transfers without other parts
    of the server running out of handles. This number is shared between all
    HTTP/2 connections in the same server process.
    The default is set to 2. With H2MaxWorkers on most platforms/mpms
    defaulting to 25 that gives a maximum of 50 handles that can be
invoved in
    transfers.
    I think I have to write a blog post one day of how this works and
affects
    performance. tl;dr the more handles http2 may use, the faster static
files
    can be served.

See also "H2SessionExtraFiles " on
http://httpd.apache.org/docs/current/en/mod/mod_http2.html

For questions on mod_http2 you might reach out to the httpd dev list.
Stefan is quite responsive to questions about mod_http2.

One might wonder though, which use case you want to support. Serving
1000 concurrent requests in a single connection creates a conflict
between fairness - answering all requests at the same pace - and
efficiency - answering them using the smallest amount of resources.

Regards,

Rainer
-------------------------------------------------------------------
List admin: https://cool.haxx.se/list/listinfo/curl-library
Etiquette: https://curl.haxx.se/mail/etiquette.html
Received on 2016-03-02