Re: Curl's network performance is slower than IDM.
Date: Tue, 16 Jun 2020 23:57:37 +0800
Daniel Stenberg <daniel_at_haxx.se> 于2020年6月16日周二 下午10:50写道：
> On Tue, 16 Jun 2020, Hongyi Zhao wrote:
> >> Are these transfer speeds somewhat stable and reliable in repeated tests?
> > I've conducted several tests at different times against this URL using both
> > curl and IDM, though the speeds may vary each time, IDM always beats curl.
> The huge difference is what makes this so weird. I mean, I wouldn't be
> surprised if a Windows-specific tool maybe can do transfers slightly faster
> somehow when tickling the OS the right way, but I can't understand what would
> make the difference that big. To me this smells like a setting/config
> somewhere that needs to be tweaked or something, rather tham just a basic
> software architectural mistake or overhead.
> Uh, but you compare against curl on Ubuntu?
Maybe I should have told the testing environments more clearly.
Anyway, I describe again on the testing environments in a more
detailed way as follows:
All testings are done on Ubuntu 20.04.
Curl is installed from the distro's repo.
I'm a registered user of IDM, and the IDM latest version is run from
within wine  git master version compiled by myself on Ubuntu 20.04.
 Wine source code is cloned from here: https://source.winehq.org/git/wine.git
> IDM is a Windows-only tool, right?
Yes, but I run it from within wine on Ubuntu 20.04, as told above.
> You're *sure* there aren't differences between the different system setups
> that could explain for (parts of) this? Wouldn't it make more sense to compare
> with curl running on the same Windows machine than runs IDM?
I don't have machine which running Microsoft Windows, so I cannot do
the testings on Windows machine.
> >> Do the same symptoms reproduce even if you don't use a SOCKS proxy?
> > It seems this URL is banned for my location. Without using the socks5 proxy,
> > the connection won't be established.
> If you try against other test servers/proxies, I presume you see similar
> differences as well?
I've tried with this URL
which don't be banned for my location without using any proxy, and the
results is the same: IDM beats curl.
> >> From my dev machine in Sweden, my curl reached 30MB/s from this URL.
> > Have you also tested with IDM for comparison just like I've done?
> I don't have a comparable Windows setup to test on. Also, this application is
> a commercial one that isn't open source so I rather avoid it for that reason
> BTW, if curl is slow then surely you could also get faster download speeds
> with other transfer tools? Have you identified any such, that perhaps runs on
To be frank, it seems there is no other tool/library on Linux which
can afford so many features as (lib)curl. I really feel so difficult
for me to find an alternative one which support multi threading (in
library mode), http(s)/socks4/5 proxies supporting, and so on.
> >> I'm not aware of any. Reading on the IDM site, it seems very content about
> >> its use of multiple connections and ranges to download data.
> > All these techniques can also be done with libcurl-multi which is one
> > of the goals of this project: https://github.com/pyIDM/PyIDM
> I'm aware. I'm just trying to make sure that we compare apples with apples.
> > Interesting. IMO, the curl itself doesn't support multi threading technique,
> It's not so much a threading issue as multiple TCP connections
Still interesting again. Is there any example for libcurl to establish
multiple TCP connections within one threading?
> that's going to make a difference.
> > and for IDM, I set the max. conn. number = 1 for testings.
> > But here, you told `a single TCP stream `. I still cannot figure out how to
> > check or ensure I'm meeting this condition. Any more hints?
> I can only think of using wireshark or something to verify what's going on
> under the hood but I guess that we also have no actual reason to suspect that
> "conn. number = 1" isn't actually limiting the number of connections to 1.
So, here, by saying *connections*, you still refer to TCP connections,
instead of the number of thread/process.
-- Hongyi Zhao <hongyi.zhao_at_gmail.com> ----------------------------------------------------------- Unsubscribe: https://cool.haxx.se/list/listinfo/curl-users Etiquette: https://curl.haxx.se/mail/etiquette.htmlReceived on 2020-06-16