cURL / Mailing Lists / curl-library / Single Mail

curl-library

RE: Problem with content-length and content-encoding: gzip

From: Daniel Stenberg <daniel_at_haxx.se>
Date: Wed, 25 Jun 2003 11:38:46 +0200 (CEST)

On Wed, 25 Jun 2003, Erik Rehrmann wrote:

> > Content-Length: header that specifies the size of the content that it
> > sends. The server only sends the compressed number of bytes, and thus it
> > should say so.
>
> I agree with you, but the problem is the streaming sequence. The servers
> transport layer gets a result from the application handler containing a
> complete xml document. The content-length is set to the size of the result.
> Now the header is written uncompressed and the whole data area is sent to
> the client via a gzip stream. Unfortunately there is no way to get the size
> of the compressed data before the complete resulting data is streamed to the
> client, at least not without causing performance troubles by compressing the
> whole data in memory first.

You're right, and I believe that Apache also sends the Content-Length of the
uncompressed data (someone else stated that in a totally different context in
a mail the other day). The solution is of course (in the server-end) that this
kind of contents should be sent chunked, and not use the Content-Length (for
HTTP 1.1 clients).

> > I'm not really sure how to work this out. Any suggestions anyone?
>
> Why not using the resulting decompressed number of bytes provided by zlib to
> check against the provided content-length (if content-encoding = gzip) of
> the header?

Yes, I figure that's the way we need to do, but that also makes us assume that
all servers will send the Content-Length: that way. Which may or may not be
true.

So, the bytecounter needs to be set to the uncompressed size but we need to
store this in an extra counter somehow to not make the progress data using the
uncompressed size as well.

-- 
 Daniel Stenberg -- curl: been grokking URLs since 1998
-------------------------------------------------------
This SF.Net email is sponsored by: INetU
Attention Web Developers & Consultants: Become An INetU Hosting Partner.
Refer Dedicated Servers. We Manage Them. You Get 10% Monthly Commission!
INetU Dedicated Managed Hosting http://www.inetu.net/partner/index.php
Received on 2003-06-25