Buy commercial curl support from WolfSSL. We help you work
out your issues, debug your libcurl applications, use the API, port to new
platforms, add new features and more. With a team lead by the curl founder
himself.
Re: Critique our use of curl; and HTTP/2, /3, multiplexing & pipelining questions
- Contemporary messages sorted: [ by date ] [ by thread ] [ by subject ] [ by author ] [ by messages with attachments ]
From: Daniel Stenberg via curl-library <curl-library_at_lists.haxx.se>
Date: Wed, 22 Feb 2023 17:01:00 +0100 (CET)
On Tue, 21 Feb 2023, Richard W.M. Jones via curl-library wrote:
> Firstly, I don't understand if the multi interface would actually help us
> here. Because nbdkit gives us lots of threads and expects an NBD request to
> be processed synchronously on that thread, using the easy interface is a
> natural .. easy(!) .. fit.
Doing multiple transfers using the multi interface instead of separate threads
using the easy interface has some pros and some cons.
Pros:
- shared pools and caches
- you can multiplex transfers with h2/h3
Cons:
- multiple threads running on multiple cores can perform faster in case you
are CPU bound
> A second thing I'm unclear about with multi is whether the individual easy
> handles which are added are related in any way -- eg. if they all share the
> same TCP connection to the web server?
The *can* share the same conncetion with HTTP/2 and HTTP/3 multiplexing, yes.
> Reading the page makes me think this is not the case, the multi interface is
> just a way to group easy handles for the purposes of using a select/poll or
> event-driven API, and apart from that there is no relationship.
They also share connection pool, DNS cache, CA cert cache etc.
> The third and main concern is whether we are using curl most efficiently.
> In particular, whether we are using HTTP/2 (and in future HTTP/3) as
> efficiently as we could be (eg. exploiting multiplexing).
For multiplexing you *must* use the multi interface and add multiple transfers
to it.
> I notice that HTTP/1.1-style pipelining was removed from curl, and I suppose
> HTTP/2 multiplexing is meant to replace this. However since we are using
> the easy interface and doing everything synchronously, it's my understanding
> that we are not exploiting multiplexing, unless curl itself does something
> clever internally.
It can't do anything clever when for synchronous serial transfers. Then you
can't do multiplexing.
Date: Wed, 22 Feb 2023 17:01:00 +0100 (CET)
On Tue, 21 Feb 2023, Richard W.M. Jones via curl-library wrote:
> Firstly, I don't understand if the multi interface would actually help us
> here. Because nbdkit gives us lots of threads and expects an NBD request to
> be processed synchronously on that thread, using the easy interface is a
> natural .. easy(!) .. fit.
Doing multiple transfers using the multi interface instead of separate threads
using the easy interface has some pros and some cons.
Pros:
- shared pools and caches
- you can multiplex transfers with h2/h3
Cons:
- multiple threads running on multiple cores can perform faster in case you
are CPU bound
> A second thing I'm unclear about with multi is whether the individual easy
> handles which are added are related in any way -- eg. if they all share the
> same TCP connection to the web server?
The *can* share the same conncetion with HTTP/2 and HTTP/3 multiplexing, yes.
> Reading the page makes me think this is not the case, the multi interface is
> just a way to group easy handles for the purposes of using a select/poll or
> event-driven API, and apart from that there is no relationship.
They also share connection pool, DNS cache, CA cert cache etc.
> The third and main concern is whether we are using curl most efficiently.
> In particular, whether we are using HTTP/2 (and in future HTTP/3) as
> efficiently as we could be (eg. exploiting multiplexing).
For multiplexing you *must* use the multi interface and add multiple transfers
to it.
> I notice that HTTP/1.1-style pipelining was removed from curl, and I suppose
> HTTP/2 multiplexing is meant to replace this. However since we are using
> the easy interface and doing everything synchronously, it's my understanding
> that we are not exploiting multiplexing, unless curl itself does something
> clever internally.
It can't do anything clever when for synchronous serial transfers. Then you
can't do multiplexing.
-- / daniel.haxx.se | Commercial curl support up to 24x7 is available! | Private help, bug fixes, support, ports, new features | https://curl.se/support.html -- Unsubscribe: https://lists.haxx.se/mailman/listinfo/curl-library Etiquette: https://curl.se/mail/etiquette.htmlReceived on 2023-02-22