Ok, so let's put Linux to a test.
0. Patch CURL:
#define UPLOAD_BUFSIZE (1<<19)
1. Create a large file. 1 TB looks good:
dd if=/dev/zero of=testfile bs=1 count=0 seek=1T
The file is sparse, so disk operations won't block us.
2. Upload to /dev/null:
curl -u user:pass -k sftp://127.0.0.1/dev/null -T testfile
  % Total % Received % Xferd Average Speed Time Time Time Current
                                 Dload Upload Total Spent Left Speed
  0 1024G 0 0 0 576M 0 34.5M 8:26:28 0:00:16 8:26:12 31.6M^C

Interrupted after a few sec. 30-40MB/sec, decent speed. Curl uses 100%
CPU, SSHD uses 60%.
3. Add some latency:
sudo tc qdisc add dev lo root netem delay 40ms
(note: this latency is only half RTT, so in this case RTT=80ms
4. Repeat test:
  % Total % Received % Xferd Average Speed Time Time Time Current
                                 Dload Upload Total Spent Left Speed
  0 1024G 0 0 0 608M 0 14.8M 19:36:56 0:00:40 19:36:16 15.1M^C
Very stable speed, 15.1-15.3MB/s. Curl uses 60% CPU, SSHD uses 30%.
4a. optionally inspect traffic:
sudo tcpdump -i lo -w dump tcp port 22
View results:
tcpdump -r dump | less
note: tcpdump doesn't play well with emulated delays, so reading the
dump is tricky.
5. Clean up
sudo tc qdisc del dev lo root netem

TCP window is not a problem, and SSH window doesn't seem to be the
problem either. There is definitely still some room for improvement.
-------------------------------------------------------------------
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library
Etiquette: https://curl.haxx.se/mail/etiquette.html
Received on 2018-08-10