Hello Daniel,

> > If you like to run me some more experiements, I can do that. I just
> > compile curl with a higher SO_SNDBUF / SO_RCVBUF, fire up a ec2 AWS
> > instance with windows server 2016 somewhere in the world with a big pipe
> > and high latency and download something from my well connected
> > webserver.

> Very interesting. It might confirm that the OS level "auto-tuning" is
> disabled for non-blocking sockets.

I see.

> Have you tried to do something like Daniel's upload "auto-tuning" patch but
> for the download buffers [*]? Or maybe that's not as easily done for
> downloads...

> [*] = https://github.com/curl/curl/pull/2762

No, I have not. I just tried to calculate the window size manually using
the throughput and latency and adapt. Can you get me a patch ready? I
can test it on a high latency big pipe link and report back.

I just fired up a AWS VM in Canada and downloaded a file from my
WebServer in Germany. It is not as bad as I thought, but it could be
improved.

C:\Users\Administrator\Downloads>curl -k https://glanzmann.de/path/to/Fedora-20-x86_64-netinst.iso --output fedora.iso
  % Total % Received % Xferd Average Speed Time Time Time Current
                                 Dload Upload Total Spent Left Speed
100 321M 100 321M 0 0 27.0M 0 0:00:11 0:00:11 --:--:-- 28.9M

This is with 92ms latency on a gigabit link Canada <=> Germany.

But honestly I wonder why it is so fast, because I don't see where you set in
curl the SO_RCVBUF. I see the workaround for SO_SNDBUF but this should
not matter here. On the other site was an nginx with Linux:

net.ipv4.tcp_wmem = 4096 16384 4194304

I also tried to increase it to

(infra) [~] sysctl net.ipv4.tcp_wmem='4096 16384 13000000'

But same result. However curl is already pretty good at it.

If I download the same file with internetexplorer, I get 2.6 MB/s.

What I do _not_get is that according to wireshark the tcp window is 32
KB for the download direction (Germany > Canada) but 15 MB for Canada >
Germany. Very strange. If I should do some more tests, let me know. I
used git HEAD (curl-7_61_0-53-g81be25487) for that.

Download Direction: https://thomas.glanzmann.de/static/screenshot-x1-2018-08-03-23_21_22.png
Way back: https://thomas.glanzmann.de/static/screenshot-x1-2018-08-03-23_21_16.png

88.198.249.254 is in Germany.

Cheers,
        Thomas
-------------------------------------------------------------------
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library
Etiquette: https://curl.haxx.se/mail/etiquette.html
Received on 2018-08-03