Re: Curl's network performance is slower than IDM.

Daniel Stenberg <daniel_at_haxx.se> 于2020年6月16日周二 上午5:44写道：
>
> On Sun, 14 Jun 2020, Hongyi Zhao via curl-users wrote:
>
> > On Ubuntu 20.04
>
> I presume that's curl 7.68.0 ?
>
> > $ curl -L -x socks5h://127.0.0.1:10119
> > http://speedtest-ny.turnkeyinternet.net/1000mb.bin >/dev/null
> > % Total % Received % Xferd Average Speed Time Time Time Current
> > Dload Upload Total Spent Left Speed
> > 46 1024M 46 475M 0 0 9.7M 0 0:01:44 0:00:48 0:00:56 11.1M^C
> >
> > The IDM gives around 16 ~ 20 MB/s speed.
>
> Are these transfer speeds somewhat stable and reliable in repeated tests?

I've conducted several tests at different times against this URL using
both curl and IDM, though the speeds may vary each time, IDM always
beats curl.

>
> Do the same symptoms reproduce even if you don't use a SOCKS proxy?

It seems this URL is banned for my location. Without using the socks5
proxy, the connection won't be established.

> What's the theoretical maximum speed for you against this URL?

Maximum theoretical upper limit speed promised from my ISP is around
300 Mbps. But I really don't know the theoretical maximum speed for me
against this URL considering that there are many factors which will
affect the real speed up limit from my location to this URL.

> From my dev machine in
> Sweden, my curl reached 30MB/s from this URL.

Have you also tested with IDM for comparison just like I've done?

>
> > I'm confused on the above results. Why IDM shows a higher network
> > performance than curl? Are there some bottlenecks/glitches/bugs in curl's
> > network transfer algorithms?
>
> I'm not aware of any. Reading on the IDM site, it seems very content about its
> use of multiple connections and ranges to download data.

All these techniques can also be done with libcurl-multi which is one
of the goals of this project: https://github.com/pyIDM/PyIDM.

Another wonderful downloading technique used by IDM is its dynamic
file segmentation technology described here:
http://www.internetdownloadmanager.com/support/segmentation.html. I've
discussed this technique with the developer of PyIDM, and he has
implemented the similar method for PyIDM, see here for more info:
https://github.com/pyIDM/PyIDM/issues/97.

But, even I and the developer of PyIDM do our bast for testing and
boosting the speed performance of PyIDM, it still defeated by IDM.
Considering that PyIDM is based on pycurl, hence libcurl, so I ask
this question here.

Another consideration is that PyIDM currently not use the
libcurl-multi interface due that the author of PyIDM has some
difficulties to figure out till now. See here for more info on this
topic: https://github.com/pyIDM/PyIDM/issues/112.

> Have you verified that your test case with that actually only uses a single TCP stream?

Interesting. IMO, the curl itself doesn't support multi threading
technique, and for IDM, I set the max. conn. number = 1 for testings.

But here, you told `a single TCP stream `. I still cannot figure out
how to check or ensure I'm meeting this condition. Any more hints?

> Just sothat we compare the same things.

Regards,

-- 
Hongyi Zhao <hongyi.zhao_at_gmail.com>
-----------------------------------------------------------
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-users
Etiquette:   https://curl.haxx.se/mail/etiquette.html