On Fri, 26 Jul 2013, Daniel Stenberg wrote:

tl;dr: I fixed the problem by setting CURLOPT_TCP_NODELAY.

> Using strace -T, I can also see that the poll() call actually takes that
> long:
>
> 23:58:38.816031 poll([{fd=3, events=POLLIN}], 1, 1002) = 1 ([{fd=3,
> revents=POLLIN}]) <0.037972>

This was the clue I needed. I fired up wireshark and just traced the TCP
connection, and yeah, due to the Nagle slow start algorithm the peer quite
simply waits a while until it ACKs the package. And with this extremly small
package sizes in the tests, those 30-40 milliseconds per request gives an
enormous impact.

So, I added CURLOPT_TCP_NODELAY in my test program and wham, the code flies...
I didn't compare it with the fixed-string version now, but I didn't feel the
need once I just analyzed in wireshark how the packet flow looks like now.
There's no more delayed ACK now and the packet frequency is thus MUCH higher
and I would expect that it is at least near the fixed-string version.

I didn't dive any deeper into this, as I believe this was the reason. Quite
simply due t the nature of HTTP/TCP, and we already have an option to work
around this effect.

I do have the captures stored if anyone is very interested in doing further
analyzes.

I consider this case closed unless someone has a compelling reason not to!

-- 
  / daniel.haxx.se
-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-library
Etiquette:  http://curl.haxx.se/mail/etiquette.html