Bugs item #3076808, was opened at 2010-09-27 23:08
Message generated for change (Comment added) made by bagder
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=100976&aid=3076808&group_id=976

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: http
Group: wrong behaviour
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Chris Smowton ()
Assigned to: Daniel Stenberg (bagder)
Summary: Requests fail silently following a 416 error

Initial Comment:
curl 7.19.7 (i486-pc-linux-gnu) libcurl/7.19.7 OpenSSL/0.9.8k zlib/1.2.3.3 libidn/1.15

The problem: I'm using libcurl to conduct a group of transfers with Range: headers. These might result in HTTP error 416: Requested Range Not Satisfiable.

The author of lib/http.c's comments suggest s/he thought that 416 responses cannot have a body, and carry a Content-Length which describes the length of the real document. However, this isn't the case. The RFC says nothing about 416 under Content-Length, but does state that Content-Range may be used for this purpose. In a few places it mentions response codes which preclude a body; it never includes 416 in such a list.

Real servers in fact *do* include a body, and don't treat Content-Length in an unusual way. Specifically:
* Apache/2.3.8 does not give a Content-Length at all, and includes an error document with chunked encoding.
* Microsoft IIS 7.5 gives a Content-Length describing the length of the error document, and uses Content-Range to indicate the valid ranges which could have been requested as suggested by the RFC
* CherryPy 3.1.2 (a Python WGSI server) uses Content-Length and an error document, but doesn't include a Content-Range header.

Full conversations with these servers are attached.

Ordinarily this is just annoying, but it becomes a serious problem when using libcurl's multi-handles and a server which may delay appreciably between sending its headers and its error document. As far as I can tell from packet traces, what's happening is:

* I add two easy-handles to a multi-handle which make requests of the same server.
* libcurl connects and issues the first request
* The server sends the headers for the 416 reply
* libcurl ignores the Content-Length header, assuming it to be irrelevant and that 416s don't have bodies
* libcurl re-uses the socket to send the second easy-handle's request
* The server sends the error document for the first 416, followed by the headers and document for the second request
* libcurl becomes confused as it gets HTML where it was expecting an HTTP reply

I'm not sure exactly what happens at the end, as I never see the second request complete. However, packet traces show that the second request is definitely sent before the first error document has arrived, and the code comments indicating false expectations about 416 replies suggest this is the cause. I'll try to track it down more carefully soon.

----------------------------------------------------------------------

>Comment By: Daniel Stenberg (bagder)
Date: 2010-09-27 23:23

Message:
Thanks for the detailed and well expressed report.

I can only explain this weirdness as a confused mixup of Content-Range and
Content-Length somehow (done by me). This change was introduced in commit
a5e22867c7ec back in October 2003 but as you say there's nothing in the
HTTP spec that backs this up.

We need to clear this up...

----------------------------------------------------------------------

You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=100976&aid=3076808&group_id=976
Received on 2010-09-27