curl-library
RE: Massive HTTP/2 parallel requests
Date: Wed, 2 Mar 2016 15:30:28 +0100
The results of curl —version are:
curl 7.47.1 (x86_64-apple-darwin15.3.0) libcurl/7.47.1 OpenSSL/1.0.2f zlib/1.2.5 nghttp2/1.7.0
Protocols: dict file ftp ftps gopher http https imap imaps ldap ldaps pop3 pop3s rtsp smb smbs smtp smtps telnet tftp
Features: IPv6 Largefile NTLM NTLM_WB SSL libz TLS-SRP HTTP2 UnixSockets
Concerning the server, I’m running an apache with mod_http2 enabled and the following HTTP/2-related configuration:
LoadModule http2_module modules/mod_http2.so
<IfModule http2_module>
Protocols h2c http/1.1
LogLevel http2:info
</IfModule>
I performed this very morning another test adding the following line: H2MaxSessionStreams 1000000
I was just to make sure that the number of streams is higher than the number of parallel downloads for my test (1024). I realised looking at the logs that all my requests fit in the pipe, but still files are downloaded in 64-stream blocks! If I try with normal HTPP/1.1 connections the total test is around 20 times faster than this way.
On a second test I set the following parameters:
curl_multi_setopt(multi_handle, CURLMOPT_MAX_TOTAL_CONNECTIONS, 10);
curl_multi_setopt(multi_handle, CURLMOPT_MAX_HOST_CONNECTIONS, 10);
And what happens is quite annoying: it opened the connection 0 for the file 1, and consecutive connections 1-9 for files 2-10. The rest 11-1024 where attached to connection 0, so that when connections 1-9 finished (with only one request) they were not reused.
Any clues?
José
> Date: Wed, 2 Mar 2016 11:42:59 +0000
> From: Lucas Pardue <Lucas.Pardue_at_bbc.co.uk>
> To: libcurl development <curl-library_at_cool.haxx.se>
> Subject: RE: Massive HTTP/2 parallel requests
> Message-ID: <7CF7F94CB496BF4FAB1676F375F9666A2A79C174_at_bgb01xud1012>
> Content-Type: text/plain; charset="utf-8"
>
> Hello,
>
> What version of libcurl are you using? There were some issues with libcurl's multiplexing reported in January that were fixed, see https://curl.haxx.se/mail/lib-2016-01/0031.html.
>
> Note that the maximum number of concurrent streams is directional (https://http2.github.io/http2-spec/#SETTINGS_MAX_CONCURRENT_STREAMS). It sounds like perhaps the server is restricting you to only 64 concurrent client-initiated streams per connection. Can you tell us the value the server is using? If you don't have access to server config a packet trace of the SETTINGS frame (or dump in client code) will let us know.
>
> Regards
> Lucas
-------------------------------------------------------------------
List admin: https://cool.haxx.se/list/listinfo/curl-library
Etiquette: https://curl.haxx.se/mail/etiquette.html
- application/pgp-signature attachment: Message signed with OpenPGP using GPGMail