A question about Z_BUF_ERROR being ignored

Hello!

When debugging some issue regarding possibly broken gzipped HTTP response in
Go[1], I noticed that curl has no problems decompressing it, and returns
successful exit code.

    $ curl -s https://regexr.com/ --compressed >/dev/null; echo $?
    0

However, the compressed data appears to be broken, as interpreted by
other decompressors. Most of them return all the data successfully (i.e.
up to and including the closing </html>), and report the error
afterwards. Busybox zcat loses some data at the end, though.

    $ curl -s https://regexr.com/ --header 'Accept-Encoding: gzip' | zcat >/dev/null; echo $?
    gzip: stdin: unexpected end of file
    1

    $ curl -s https://regexr.com/ --header 'Accept-Encoding: gzip' | zstdcat >/dev/null; echo $?
    zstd: /*stdin*\: premature gz end
    1

Go also reports this as an error[1].

I've found the place where libcurl ignores the error: [2]

Does anyone has a clue what's going on? Does curl erroneously not report
a broken file? Or it's some known workaround for broken web servers?

I've searched the mailing list archive for Z_BUF_ERROR-related
workarounds, but it appears that those workarounds are not in the code
anymore (and they were more complicated than ignoring the error anyway).

[1] https://github.com/gocolly/colly/issues/511
[2] https://github.com/curl/curl/blob/8df455479f8801bbebad8839fc96abbffa711603/lib/content_encoding.c#L222-L224
[3] https://github.com/facebook/zstd/blob/4c144cf9c5fc396199bfb4b527ad7d213d194963/programs/fileio.c#L2021-L2047
-------------------------------------------------------------------
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library
Etiquette: https://curl.haxx.se/mail/etiquette.html
Received on 2020-06-15