cURL / Mailing Lists / curl-library / Single Mail

curl-library

Massive HTTP/2 parallel requests

From: Molina <jose.molina_at_cern.ch>
Date: Wed, 2 Mar 2016 10:46:56 +0100

Hello,

I’m working in a project that downloads a decent amount of files simultaneously, with an average size of 512KB. We’ve been using CURL with HTTP/1.1 requests so far, and I’ve been modifying the code to adapt it to HTTP/2.0. However, when trying to simultaneously download 1024 files for testing I realised only 64 where actually active, while the rest were idle. I was using only one multiplexed connection with CURLOPT_PIPEWAIT enabled.

Secondly, just in order to try I set the following flags:

  curl_multi_setopt(multi_handle, CURLMOPT_MAX_TOTAL_CONNECTIONS , 1024);
  curl_multi_setopt(multi_handle, CURLMOPT_MAX_HOST_CONNECTIONS, 1024);

Which apparently enabled multiple 64-file-wide HTTP/2.0 connections, where each one received the corresponding “101, switching protocols” answer.
Is this limit something desirable or convenient for the performance of HTTP/2.0? Would you have an advice for this use case?

Thanks a lot in advance, and best regards.
José Molina

-------------------------------------------------------------------
List admin: https://cool.haxx.se/list/listinfo/curl-library
Etiquette: https://curl.haxx.se/mail/etiquette.html

Received on 2016-03-02