New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
HTTP response could not be read if using POST on Windows #657
Comments
Thanks! I don't get why just those two states are important. Shouldn't we then read data as soon as possible in any state if we want to get the data sent before the server closes the connection (sending a FIN or RST I guess?)? But then: aren't we already? Can you link to more info about this windows bug? Why aren't we seeing more problems with this bug if it really is this blatantly stupid? |
Actually in Windows case you better to always check for incoming data when trying to send something. But practically, in case of HTTP(S) you must check for incoming data when sending request.
In case of HTTP POST and PUT, send() can be called multiple time before any single recv() is attempted.
Try my testsuite from the Issue description. It's multiplatfrom and show clearly situations where this bug is not arise and where it's in action.
Combination of several conditions is required:
Seems than final delay is only required for local loopback, when using real networks only 1-3 is enough to trigger the bug. libcurl usually don't attempt to send data when expecting some incoming data from remote. |
I think you should report this to Microsoft and see what they have to say. |
I think that they reply similar to |
Do you mean you think they will reply that way or you think they did reply that way? Where is the reply, who did you contact and how do you know there's no chance? I think the delay is key here. In Windows 7 x64 (SP1 Enterprise) I changed your server example to shutdown via
As long as server send is shutdown before server receive the client will finish without any errors. The doc says:
I think if you want to pursue this further you will first need to monitor in Wireshark or something to figure out exactly what's happening. Also there's a separate doc for graceful shutdown you may want to read. |
Just checked - only error in last test is gone, not error in test before last one.
However, it doesn't change the fact that WinSock destroy received data if send() function is used. If you don't call send(), WinSock will hold received data until if will be read by recv() even if connection is closed. But as soon as you call send() for same socket - WinSock will not return any data for subsequent recv(). This is very common for Web servers: in case of some internal error or other inability to process request - reply with short error description and close connection. Web server can't wait for each client to read response - it will be too costly. The problem is only on WinSock receiver side. If you run server_part on Windows and client_part on any other OS (tested Linuxes, FreeBSD, OS X, Solaris) then client will read all received data in all cases. |
By default WinSock use "graceful shutdown", from MSDN closesocket():
|
WinSock destroys recv() buffer if sendf() is failed. As result - server response may be lost if server sent it while curl is still sending request. This behavior noticeable on HTTP server short replies if libcurl use several send() for request (usually for POST request). To workaround this problem, libcurl use recv() before every sendf() and keeps received data in intermediate buffer for further processing. Bug: curlgh-657
WinSock destroys recv() buffer if send() is failed. As result - server response may be lost if server sent it while curl is still sending request. This behavior noticeable on HTTP server short replies if libcurl use several send() for request (usually for POST request). To workaround this problem, libcurl use recv() before every send() and keeps received data in intermediate buffer for further processing. Bug: curlgh-657
response when POST request is used with slow read callback function. Test should check for presence of WinSock bug curlgh-657 and ability to workaround it curlgh-668. Bug: curlgh-657 Workaround: curlgh-668
WinSock destroys recv() buffer if send() is failed. As result - server response may be lost if server sent it while curl is still sending request. This behavior noticeable on HTTP server short replies if libcurl use several send() for request (usually for POST request). To workaround this problem, libcurl use recv() before every send() and keeps received data in intermediate buffer for further processing. Bug: curlgh-657
This is actually a WinSock bug, but workaround can be implemented.
I discovered it while debugging GNU libmicrohttpd test suite.
Problem: WinSock destroy received data buffer if it detected TCP closure by remote side.
In practice:
WSAECONNABORTED
orWSAECONNRESET
. Received data is destroyed by WinSock. libcurl returns network error instead of real HTTP code.To illustrate it, I've created small test suite:
https://github.com/Karlson2k/check_system_socket_recv_last
This bug is the reason why function
testMultithreadedPostCancelPart
in https://github.com/Karlson2k/libmicrohttpd/blob/master/src/testcurl/test_post.c#L538 failed when called withFLAG_SLOW_READ
on Windows.To workaround this Windows bug, libcurl must call recv() ASAP if incoming data is detected in
CURLM_STATE_DOING
,CURLM_STATE_DO_MORE
states. Not sure that it can be easly implemented without breaking something.The text was updated successfully, but these errors were encountered: