I completely agree with you!
This is surely a non-standard implementation of a 1.1 webserver.
If I could change webserver response headers I would never bother you, but we are talking about a closed box!
I forgot to mention that Firefox can handle correctly this situation. And also the proxy(squid) itself.

When you say:
> If there's no content-length header in a HTTP 1.1 response, there are two
> other ways to "mark" the end of it: 1) send the response with
> chunked-encoding
> and thus sending the header that says so or 2) close the connection after
> the response and thus sending the header that says so.

In the second point I think you mean one of these headers:
"Connection: close"
"Proxy-Connection: close"

I read the rfc and I have to confirm that persistent connections are the default for http 1.1. For anybody interested see 8.1.2
http://www.w3.org/Protocols/rfc2616/rfc2616-sec8.html

And at this point my suggestion:
If the default behaviour is to consider the server persistent why not have a option (like --close) to let the curl/libcurl user decide the default. So having this option will send the "Connection: close" header and will expect the server to close the connection (even if it doesn't send the same header) so the library will receive the body until the server closes the connection.

I would do it myself and send a patch but I have difficulties finding where/how apply this changes.
If someone can give me some hints about this it would be great.

Thanks Daniel for you reply.
Luigi

-----Messaggio originale-----
Da: curl-users-bounces_at_cool.haxx.se [mailto:curl-users-bounces_at_cool.haxx.se] Per conto di Daniel Stenberg
Inviato: martedì 8 maggio 2007 12.45
A: the curl tool
Oggetto: Re: Curl exit when reply doesn't contain content-length

On Tue, 8 May 2007, Luigi Grilli wrote:

> When I try to connect to the server through our company proxy (using -x
> host:port option of curl) I receive correctly all reply from the webserver,
> so NO problems with proxy.

... and I figure that is because the proxy adds at least one header to make
the response fine protocol-wise...

> The problems comes when I direct connect to the webserver. If the webserver
> return a content-length header curl works perfectly. If the server doesn't
> return the content-length curl exit without waiting for the response body.

If there's no content-length header in a HTTP 1.1 response, there are two
other ways to "mark" the end of it: 1) send the response with chunked-encoding
and thus sending the header that says so or 2) close the connection after
the response and thus sending the header that says so.

How is libcurl supposed to know what your server intends to behave?

This "issue" was indeed added because we made libcurl somewhat more strictly
adhering to the HTTP spec, but we also did this because having the opposite
behaviour does indeed cause problems to those who actually assumes or depends
on this kind of behavior!

> Reading some past mails of this list I read the expected behaviour was
> for HTTP1.0 to wait for the webserver to close the connection. This is
> how other clients that works with this webserver get the reply content
> (and expect the server to close the connection).

HTTP 1.1 is persistent by default, so that sounds like a weird assumption
with the lack of such a header.

> I tried also with the --ignore-content-length option but it went event
> worst.

Well, there's no content-length header present to ignoring the absent header
shouldn't do much of a difference...

In my eyes, this is an web server bug. But sure, I realize we live in a world
that is filled with crappy software so there might be reasons for libcurl to
still act fine against such servers. I'm open for suggestions.

-- 
  Commercial curl and libcurl Technical Support: http://haxx.se/curl.html