On 02/20/2014 01:02 AM, Daniel Stenberg wrote:
> On Tue, 18 Feb 2014, Justin Karneges wrote:
>
>> I've observed that if I POST some data to a server and the server
>> responds before the data has been sent, then libcurl will receive the
>> response but wait 10 seconds before indicating completion.
>
> You use the term "Race" in the subject, is that because it doesn't
> always happen or what makes it a race?

Correct. The bug only happens if curl is unable to send its POST data
before the server replies. Sometimes curl does manage to get the data
out the door in time, and then the logs show "We are completely uploaded
and fine" and I receive CURLMSG_DONE immediately after the server responds.

>
>> Libcurl seems to recognize that a problem has happened ("stop
>> sending"), gives me the response data (writeFunction called with 13
>> bytes), but then stalls. Eventually socketFunction is invoked to say
>> the fd is no longer needed, and if I read messages after that then I
>> get CURLMSG_DONE. I am able to reproduce this issue easily, and the
>> delay is always around 10 seconds. Maybe the 10 second delay is a
>> helpful hint about where to look inside libcurl?
>
> There's no fixed 10 second timeout or delay anywhere so I don't think
> that helps us a lot. The message "stop sending" is however only saying
> that libcurl won't send any more, it will still receive the response.
> How do the response look like in this case? I mean all the response
> headers etc. How does it signal end of response?

Response looks okay. You can see in my original email the headers that
were received, and also that a 13 byte body was received, which is all
correct. My program then does not receive CURLMSG_DONE until 10 seconds
pass.

>
> You're using the multi_socket API right? I think this could be a mere
> oversight somehow for that case.

Yes I'm using the multi_socket stuff.

>
> Can you help me repeat this problem so I can check closer exactly what
> happens there?
>

In my case I am testing against an internal Apache+Django project that
requires authentication, and making a POST without proper
authentication. This causes the server app to reject the request before
looking at the request body. In fact, I've discovered that I can
reproduce this with the curl command itself:

$ time curl -d @somefile http://localhost:8000/path/
Unauthorized

real 0m10.057s
user 0m0.008s
sys 0m0.008s

So maybe the bug is not multi_socket specific. The file I'm uploading
here is around 600K. Not huge, but enough to hopefully hit TCP buffer
limits so that the server responds before curl can fully send. Note that
it is not enough to test against Apache alone, as in file-serving mode
Apache appears to always read the entire request body before responding.

However, I am now wondering if this is really an Apache/WSGI/Django bug.
I created a simple HTTP server test in python to provide the necessary
behavior:

import socket
import time

s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.bind(('', 8010))
s.listen(1)

while True:
         conn, addr = s.accept()
         print 'connected: %s' % repr(addr)
         while True:
                 data = conn.recv(1024)
                 if '\r\n\r\n' in data:
                         break
         print 'got request header'
         conn.send('HTTP/1.1 401 OK\r\nContent-Type:
text/plain\r\nTransfer-Encoding: chunked\r\nConnection:
Transfer-Encoding\r\n\r\n6\r\nhello\n\r\n')
         conn.send('0\r\n\r\n')
         conn.close()
         print 'finished'

I am unable to reproduce the bug when connecting curl to this server!
Perhaps the mysterious 10 second delay is within the Apache stack. Yikes.

Justin
-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-library
Etiquette: http://curl.haxx.se/mail/etiquette.html
Received on 2014-02-20