On Tue, May 2, 2017 at 4:42 PM, Daniel Stenberg <daniel_at_haxx.se> wrote:
> On Wed, 26 Apr 2017, Rainer Canavan wrote:
>
>> we're trying to pass (some) incoming requests to another server using
>> libcurl and the corresponding response back to the client, i.e. basically
>> like a proxy.
>>
>> For Requests with non-trivial Range:-Headers, and Accept-Encoding, if the
>> server does indeed uses gzip or compressed encoding, curl fails to
>> decompress the body because it is incomplete, and returns no content. As far
>> as I can see, no error is raised, and the request appears to be successful.
>> This can easily be reproduced with the trivial change to
>> examples/getinmemory.c shown below. I'm not sure what a correct behavior for
>> curl would be, i.e. flag this as an error, pass the "raw" response body, or
>> something completely different. I believe using CURLOPT_DEBUGFUNCTION to
>> extract the body would be an appropriate workaround.
>
>
> This is an interesting situation that's tricky to act on.
>
> Passing the first 11 bytes of the gzipped response into zlib generates no
> problems of course, and it generates no output since I assume it is mostly a
> compressed header thus far. And the functions are "streaming" so they expect
> more data to get passed in.
>
> Then when not feeding any more data into the decompressor and instead just
> closing it, will similarly not cause any problems or complaints from zlib. I
> suspected the inflateEnd(z) call in the content_encoding.c:exit_zlib() could
> be blamed since we don't check the return code there, but it always returns
> OK even for this case.
>
> It is simply put not a problem in the eyes of zlib and it uncompresses the
> incoming compressed data to zero bytes output.

I've never really thought about this as a zlib problem. In case the range starts
at zero, an attempt to decompress the content of the response seems perfectly
legitimate. If the range starts at any other offset, trying to decompress the
returned fragment(s) will predictably fail, e.g. the aforementioned example with
a range of 2-74:

curl_easy_perform() failed: Unrecognized or bad HTTP Content or
Transfer-Encoding

Since that must fail, I assumed it's all curl's fault, and it should
not even try to
de-code the reply. I'm not so sure anymore.

Rainer
-------------------------------------------------------------------
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library
Etiquette: https://curl.haxx.se/mail/etiquette.html
Received on 2017-05-02