curl-library
Massive HTTP/2 parallel requests
Date: Wed, 2 Mar 2016 10:46:56 +0100
Hello,
I’m working in a project that downloads a decent amount of files simultaneously, with an average size of 512KB. We’ve been using CURL with HTTP/1.1 requests so far, and I’ve been modifying the code to adapt it to HTTP/2.0. However, when trying to simultaneously download 1024 files for testing I realised only 64 where actually active, while the rest were idle. I was using only one multiplexed connection with CURLOPT_PIPEWAIT enabled.
Secondly, just in order to try I set the following flags:
curl_multi_setopt(multi_handle, CURLMOPT_MAX_TOTAL_CONNECTIONS , 1024);
curl_multi_setopt(multi_handle, CURLMOPT_MAX_HOST_CONNECTIONS, 1024);
Which apparently enabled multiple 64-file-wide HTTP/2.0 connections, where each one received the corresponding “101, switching protocols” answer.
Is this limit something desirable or convenient for the performance of HTTP/2.0? Would you have an advice for this use case?
Thanks a lot in advance, and best regards.
José Molina
-------------------------------------------------------------------
List admin: https://cool.haxx.se/list/listinfo/curl-library
Etiquette: https://curl.haxx.se/mail/etiquette.html
- application/pgp-signature attachment: Message signed with OpenPGP using GPGMail