On Fri, 12 Oct 2007, Alessandro Vesely wrote:

> I captured another transfer of the same 8058822 bytes, using no options, and
> obtained the same sending pattern. IMHO it happens because cURL sends one
> buffer at a time, without refilling the buffer after a partial send has
> taken place.
>
> The number of buffers sent this way is 5906 = (11+1)*491 + 9+1 + 2+2:
> 12 buffers * 491 times + 14278 =
> (11 * 1460 + 324) * 491 + 1460 * 9 + 1138,
> where 1 buffer is 0x4000 bytes, see attached file.
> (Curiously, this morning I had 10 packets more on the relevant capture file.)
>
> Refilling the buffer each time gave 5523 = 5519 + 1 + 1+2:
> 5519*1460 + 1*1082 = 8057740 + 1082 = 8058822.
>
> Thus, curl sent 383 data packets more. Assuming a constant time per packet,
> that would result in approximately a 7% increase, roughly 1 second every 14.
> That, together with the rounded time in the xferlog, more or less explains
> the timings I got. Dunno what curious TCP pattern could have been triggered
> to obtain the much more relevant difference that Ivan reported...

I'm quite amazed that the TCP layer doesn't use its own buffering to even out
this... Did you try building a curl with an "even" (for example 16*1460)
buffer size instead to see if that then makes everything fly faster and thus
really prove this theory?

But I figure a more sensible fix for this flaw is to edit the code in
lib/transfer.c around line 1400 to refill the buffer earlier than when it gets
completely empty.

I can't but to feel that it seems utterly silly that this layer of code would
need to know about and adjust to the transfer layer's MTU etc. I mean, sure
1460 bytes is for the typical TCP over IPv4 ethernet, but the lib/transfer.c
code really is completely unaware and non-caring about that...

-- 
  Commercial curl and libcurl Technical Support: http://haxx.se/curl.html