Re: HTTP/2 and multi-streams

On Wed, 15 Jan 2020, Stephane Bortzmeyer via curl-and-python wrote:

> I'm trying to parallelize requests on HTTP/2 streams (not connections).
> Whatever I do, pycurl only uses one stream per HTTP/2 connection.

I don't actually know pycurl specific stuff, but I can speak for the
underlying libcurl behavior and I believe that's more relevant in this case.

> <http://pycurl.io/docs/latest/curlmultiobject.html>
> m.setopt(pycurl.M_PIPELINING, True)

Unfortunately that documentation is not telling the whole truth as of current
behavior. Compare with the up-to-date libcurl version:

  https://curl.haxx.se/libcurl/c/CURLMOPT_PIPELINING.html

The argument used there ('True') is not actually a boolean. It is, or should
be, a 'long' and back in libcurl 7.43.0 it was made into a bitmask and a
second bit was introduced for multiplexing. Setting the value to 1 thus asks
for pipelining (only). Setting it to 3 asks for either.

Since then, libcurl has dropped support for pipelining and in modern libcurl
versions it enables multiplexing by default. You can therefore simply remove
that line and it *should* improve things. But read on, it's not over... =)

> libcurl 7.67.0

In versions 7.65.0 to and including 7.67.0, I'm a little embarassed to admit,
I caused a regression in the connection reuse logic which also makes libcurl
less likely to multiplex and more likely to prefer new connections. Fixed in
7.68.0 (with https://github.com/curl/curl/pull/4732).

That regression can be worked around a little by using
CURLMOPT_MAX_HOST_CONNECTIONS to limit how many connections per host libcurl

-- 
  / daniel.haxx.se | Commercial curl support up to 24x7 is available!
                   | Private help, bug fixes, support, ports, new features
                   | https://www.wolfssl.com/contact/
_______________________________________________
https://cool.haxx.se/cgi-bin/mailman/listinfo/curl-and-python