The issue is in the sending of the data.

    No. Time Source Destination Protocol
Info
    169 2.365119 10.20.10.14 10.20.10.193 TCP
2694 > 5401 [SYN] Seq=9966193 Ack=0 Win=8192 Len=0
    170 2.365420 10.20.10.193 10.20.10.14 TCP
5401 > 2694 [SYN, ACK] Seq=422289080 Ack=9966194 Win=17640 Len=0
    171 2.365459 10.20.10.14 10.20.10.193 TCP
2694 > 5401 [ACK] Seq=9966194 Ack=422289081 Win=8820 Len=0
    172 2.366831 10.20.10.14 10.20.10.193 SSLv3
Client Hello
    173 2.367892 10.20.10.193 10.20.10.14 SSLv3
Server Hello, Change Cipher Spec, Encrypted Handshake Message
    174 2.368265 10.20.10.14 10.20.10.193 SSLv3
Change Cipher Spec, Encrypted Handshake Message
    175 2.556502 10.20.10.193 10.20.10.14 TCP
5401 > 2694 [ACK] Seq=422289235 Ack=9966403 Win=17431 Len=0
    176 2.556569 10.20.10.14 10.20.10.193 SSLv3
Application Data, Application Data, Application Data, Application Data
    177 2.563886 10.20.10.193 10.20.10.14 SSLv3
Application Data
    178 2.564092 10.20.10.193 10.20.10.14 SSLv3
Application Data
    179 2.564117 10.20.10.14 10.20.10.193 TCP
2694 > 5401 [ACK] Seq=9966775 Ack=422289565 Win=8336 Len=0
    180 2.564343 10.20.10.14 10.20.10.193 SSLv3
Encrypted Alert
    181 2.564541 10.20.10.14 10.20.10.193 TCP
2694 > 5401 [FIN, ACK] Seq=9966812 Ack=422289565 Win=8336 Len=0
    182 2.564825 10.20.10.193 10.20.10.14 TCP
5401 > 2694 [ACK] Seq=422289565 Ack=9966813 Win=17022 Len=0
    183 2.565298 10.20.10.193 10.20.10.14 SSLv3
Encrypted Alert
    184 2.565345 10.20.10.14 10.20.10.193 TCP
2694 > 5401 [RST] Seq=9966813 Ack=422289565 Win=0 Len=0

You can see the nagle delay between packets 174 and 176. It looks as
though the 176 packet is triggered by the receipt of the 175 ACK from
the server, but I think that that is just a coincidence arising from
having windows at both ends.

The operation is simple SSL POST (yes, I was confused in my earlier
message) with a few (not very big) post fields.

I am reusing the CURL context for each iteration of the loop -- this
means that the SSL association is preserved so that the expensive public
key stuff is not done on each operation.

The trace with nagle turned off:

    No. Time Source Destination Protocol
Info
    157 0.550358 10.20.10.14 10.20.10.193 TCP
2718 > 5401 [SYN] Seq=9966408 Ack=0 Win=8192 Len=0
    158 0.550671 10.20.10.193 10.20.10.14 TCP
5401 > 2718 [SYN, ACK] Seq=598520702 Ack=9966409 Win=17640 Len=0
    159 0.550708 10.20.10.14 10.20.10.193 TCP
2718 > 5401 [ACK] Seq=9966409 Ack=598520703 Win=8820 Len=0
    160 0.552150 10.20.10.14 10.20.10.193 SSLv3
Client Hello
    161 0.553712 10.20.10.193 10.20.10.14 SSLv3
Server Hello, Change Cipher Spec, Encrypted Handshake Message
    162 0.554135 10.20.10.14 10.20.10.193 SSLv3
Change Cipher Spec, Encrypted Handshake Message
    163 0.554363 10.20.10.14 10.20.10.193 SSLv3
Application Data, Application Data
    164 0.554480 10.20.10.14 10.20.10.193 SSLv3
Application Data, Application Data
    165 0.554959 10.20.10.193 10.20.10.14 TCP
5401 > 2718 [ACK] Seq=598520857 Ack=9966990 Win=17059 Len=0
    166 0.561781 10.20.10.193 10.20.10.14 SSLv3
Application Data
    167 0.561990 10.20.10.193 10.20.10.14 SSLv3
Application Data
    168 0.562013 10.20.10.14 10.20.10.193 TCP
2718 > 5401 [ACK] Seq=9966990 Ack=598521187 Win=8336 Len=0
    169 0.562128 10.20.10.14 10.20.10.193 SSLv3
Encrypted Alert
    170 0.562341 10.20.10.14 10.20.10.193 TCP
2718 > 5401 [FIN, ACK] Seq=9967027 Ack=598521187 Win=8336 Len=0
    171 0.562619 10.20.10.193 10.20.10.14 TCP
5401 > 2718 [ACK] Seq=598521187 Ack=9967028 Win=17022 Len=0
    172 0.563091 10.20.10.193 10.20.10.14 SSLv3
Encrypted Alert
    173 0.563142 10.20.10.14 10.20.10.193 TCP
2718 > 5401 [RST] Seq=9967028 Ack=598521187 Win=0 Len=0

You can see that this is much better -- however, there are three packets
(162,163,164) (lengths 75, 202, 170) which could have been merged into a
single packet. I suspect that this sequence of writes is what triggered
Nagle.

Philip

Rick Jones wrote:

I was doing some testing with GETs of https:// <https://> urls with
libcurl under

win32. I noticed that the Nagle algorithm gets triggered on every

request leading to a 200ms response time. Turning off Nagle

(setsockopt(,,TCP_NODELAY,,)) reduces the time to around 15 ms. THis

is

a big improvement.

Was this on data coming-back from the server, or in the sending of the

GET itself? I'm guessing it was in the sending of the request to the

server in the first place. Can you provide the packet trace?

Any thoughts?

Typically, if disabling Nagle improves throughput for an HTTP*

transaction, it suggests that logically-associated data (ie the

request or the response) is being given to the transport in separate

send calls. Plusungood. The "proper" fix is to present the logically

associated data to the transport at the same time - say through

something akin to writev() if the data is in separate buffers.

rick jones

The discussion gets a bit more complicated once we are talking about

_pipelined_ requests (versus persistent), but as I recall, libcurl

isn't doing pipelined just yet.

-------------------------------------------------------

This SF.net email is sponsored by:

The Definitive IT and Networking Event. Be There!

NetWorld+Interop Las Vegas 2003 -- Register today!

http://ads.sourceforge.net/cgi-bin/redirect.pl?keyn0001en
<http://ads.sourceforge.net/cgi-bin/redirect.pl?keyn0001en>

-- 
Philip Gladstone	
Okena Inc.               781-209-3155
-------------------------------------------------------
This SF.net email is sponsored by:
The Definitive IT and Networking Event. Be There!
NetWorld+Interop Las Vegas 2003 -- Register today!
http://ads.sourceforge.net/cgi-bin/redirect.pl?keyn0001en