Hello Curl hackers,
I've been working with Curl library in C for some time, and I love it.
Everything just works.

Recently I started using Curl to upload files from Windows machines to
Amazon cloud using HTTP POST. The uploads don't perform well. They are
limited by the send buffer used by Windows.
On Windows 2008R2 (and probably on more recent versions as well) the
system only puts on the wire the amount of bytes it has buffered. This
is either what is set by SO_SNDBUF (default 8KB) or what was passed to
send (CURL_MAX_WRITE_SIZE, 16KB), whichever is higher. In case of CURL
uploads, the system sends 16KB, then waits for acknowledgement from
the other end before asking the application for the next batch. This
is readily visible in Wireshark traces.

Starting with Windows 7 / 2008R2, Windows implements send buffer
autotuning. This is well described here:
https://msdn.microsoft.com/en-us/library/windows/desktop/bb736549(v=vs.85).aspx

Theoretically on these systems the send buffer should be automatically
adjusted to optimize throughput. I run a couple experiments to confirm
that, and found that the send buffer is only adjusted if the socket is
blocking and application buffer size is reasonably large
(experiment1.c), and buffer stays at 8192 if the socket is nonblocking
regardless of the application buffer size (experiment2.c).

Since Curl is using nonblocking sockets, and switching to blocking
would break existing functionality, an alternative approach would be
to periodically run SIO_IDEAL_SEND_BACKLOG_QUERY and pass the result
to SO_SNDBUF. This will work on Windows Vista SP1 / 2008 SP1 and
newer, and fail silently on older Windows versions.

I attached a proof-of-concept patch (autotuning.patch), which requires
a reasonably recent SDK. With this patch in place I am no longer
seeing upload pauses, and Windows is able to saturate the link even on
high-bandwidth high-latency networks.

Disclaimer: I only tested this patch on 32bit build under Visual
Studio 2010 (I had to #define _WIN32_WINNT 0x0601 in order to get
access to SIO_IDEAL_SEND_BACKLOG_QUERY).
64bit build currently fails on my machine
(..\builds\libcurl-vc-x64-release-dll-ipv6-sspi-winssl-obj-lib/file.obj
: fatal error LNK1112: module machine type 'X86' conflicts with target
machine type 'x64'), I don't think this is related to my patch, but
I'll keep digging.

Let me know if there are chances for this patch to make its way to the
next release.
Thanks,
Daniel

-------------------------------------------------------------------
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library
Etiquette: https://curl.haxx.se/mail/etiquette.html