Re: Curl's network performance is slower than IDM.

On Tue, 16 Jun 2020, Hongyi Zhao wrote:

> the IDM latest version is run from within wine [1] git master version
> compiled by myself on Ubuntu 20.04.

Oh ok, wow.

> the results is the same: IDM beats curl.

Presumably also by a very large margin?

>> BTW, if curl is slow then surely you could also get faster download speeds
>> with other transfer tools? Have you identified any such, that perhaps runs
>> on Linux?
>
> To be frank, it seems there is no other tool/library on Linux which can
> afford so many features as (lib)curl.

Well, neither does IDM. But there are plenty of tools and libraries that can
download a URL from the Internet and that's the operation we're comparing
here. wget is a popular one for example. Does wget also beat curl in this
setup by a large margin?

>> It's not so much a threading issue as multiple TCP connections
>
> Still interesting again. Is there any example for libcurl to establish
> multiple TCP connections within one threading?

The libcurl multi interface is providing exactly that ability. Which btw you
can use with the cmdline tool if you do -Z transfers.

> So, here, by saying *connections*, you still refer to TCP connections,
> instead of the number of thread/process.

Yes. Because what transfer tools usually find that they benfit from is to open
multiple connections, as doing many connections can often get a larger total
bandwidth than a single connection does (on the expense of other traffic on
the network of course) since TCP connections are designed to be "fair" and
allow others to also use the network. Also, middle boxes are known to exist to
limit and restrict traffic on a per-connection basis.

The network doesn't know how many threads or process you run in your machine
so as long as your code manages to read/write the sockets at a sufficiently
high speed within a single thread the number of threads will have zero effect
on that.

-- 
  / daniel.haxx.se | Commercial curl support up to 24x7 is available!
                   | Private help, bug fixes, support, ports, new features
                   | https://www.wolfssl.com/contact/
-----------------------------------------------------------
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-users
Etiquette:   https://curl.haxx.se/mail/etiquette.html