cURL / Mailing Lists / curl-library / Single Mail

curl-library

Re: The curl library finishes reading if Content-Length or Transfer-Encoding headers are not present

From: Tor Arntsen <tor_at_spacetec.no>
Date: Fri, 26 Mar 2010 17:54:05 +0100

On 26/03/2010, Michael Semenov <micsemen_at_cisco.com> wrote:
> Hi guys,
>
> I am from the XML Engineering of Cisco Systems Inc. We are using curl 7.10.1

That is a really, really old version. As far as I can remember (as a
user of libcurl in our software) it has numerous problems, including
memory leaks and larg-file problems (at the
just-a-few-hundred-megabytes stage even).

> as part of our product. If the incoming message doesn't have Content-Length
> or Transfer-Encoding headers, curl can't determine length of the message and
> doesn't read it. But according to section 4.4 (Message Length) of RFC 2616,
> message length can be determined by closing the server connection if
> "Transfer-Encoding" and "Content-Length" headers are not specified.
>
> We tried to build product with curl 7.19.5(and afaik there is newest one)
> and the issue disappeared. Unfortunately, we can not just upgrade curl to
> 7.19.5 because of modifications in curl code related to our product.
>
> We were not able to found this bug in your bug
> tracker(http://sourceforge.net/tracker/?group_id=976
> <http://sourceforge.net/tracker/?group_id=976&atid=100976> &atid=100976).
> Could you please point to bug that resolves the described issue in curl?
> Place in code that is responsible for this behavior will also help.

I would urge to you try to move to a recent version. 7.20.0 is the
current version.

It's just possible that the fix you talk about is one that was
introduced in 7.16.4 though, and if so, it should be the one attached.

commit 974fa1242aa9e7a95c687cb1d074240c53ffb33b

The core of it is:
diff --git a/lib/transfer.c b/lib/transfer.c
index 703f408..35e8192 100644
--- a/lib/transfer.c
+++ b/lib/transfer.c
@@ -577,11 +577,12 @@ CURLcode Curl_readwrite(struct connectdata *conn,
                    (k->httpversion >= 11) )
                   /* On HTTP 1.1, when connection is not to get closed, but no
                      Content-Length nor Content-Encoding chunked have been
- received, there is no body in this response. We don't set
- stop_reading TRUE since that would also prevent necessary
- authentication actions to take place. */
- conn->bits.no_body = TRUE;
-
+ received, according to RFC2616 section 4.4 point 5, we
+ assume that the server will close the connection to
+ signal the end of the document. */
+ infof(data, "no chunk, no close, no size. Assume close to "
+ "signal end\n");
+ conn->bits.close = TRUE;
               }

               if (417 == k->httpcode) {

-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-library
Etiquette: http://curl.haxx.se/mail/etiquette.html

Received on 2010-03-26