On Sat, Apr 20, 2019 at 4:31 PM Ray Satiro via curl-library
<curl-library_at_cool.haxx.se> wrote:
>
> On 4/20/2019 3:12 AM, Ray Satiro wrote:
>
> On 4/19/2019 5:01 PM, Dmitri Trembovetski via curl-library wrote:
>
> As with my earlier email to this list, the configuration is
> self-compiled libcurl with HTTP/2 support (openssl, nghttp2) for my
> project.
> curl 7.64.1
> nghttp2 v1.37.0
> openssl 1.1.0
> I've compiled and run the same app on MacOSX, Android (x64), and Linux with
> the same results.
>
> It uses platform zlib for gzip encoding on MacOSX (compatibility
> version 1.0.0, current version 1.2.11), on linux zlib was 1.2.8
> (libz.so.1.2.8), and on Android who knows, so it doesn't look like
> zlib version is the issue.
>
> I'm using multi interface.
>
> The problem is that if I enable gzip encoding
> (curl_easy_setopt(handle, CURLOPT_ACCEPT_ENCODING, "gzip")) my GET
> requests fail with CURLE_WRITE_ERROR.
> Note that the error is not caused by my WRITE callback's return
> value, it originates from process_trailer() function in
> lib/content_encoding.c:
> static CURLcode process_trailer(struct connectdata *conn, zlib_params *zp)
> {
> ...
> z->next_in += len;
> if(z->avail_in) {
> result = CURLE_WRITE_ERROR; <<< here
> }
>
> This is before my WRITE callback is invoked for this batch of data.
>
> This happens with HTTP1.1 and HTTP/2 (so unrelated to my earlier
> issue that I asked about on this list).
>
> It doesn't fail on every GET request, looks like only if the request
> has a few tens of KB of data to read.
>
> The interesting thing is that it worked with earlier curl version
> (7.56.1, I haven't bisected further) in the same exact environments,
> with the same server, both with HTTP 1.1 and /2 (HTTP/2 didn't work
> for me on POSTs on 7.56.1 but GETs were fine).
>
> Attaching relevant part of the log (taken on MacOSX) with debug
> tracing enabled and a diff that I applied for my very sophisticated
> debugging. In the log 0x7f9c79813c00 is the handle which failed.
>
> Has something changed between those curl versions? Perhaps some
> validation was added or something?
>
> Possibly known bug 1.7 "Deflate error after all content was received"
> [1]. Starting in 7.58.0 there was additional validation added to check
> for unparseable trailer bytes [2]. Later in 7.61.1 it was relaxed a
> little [3].
>
> It's interesting that it fails only on certain body sizes. Given that
> it's not hitting the writefunction we can reconstruct it from the
> trace.log and then try playing it back. At first glance there's some
> null bytes at the end, so that may be something.
>
> [1]: https://github.com/curl/curl/blob/curl-7_64_1/docs/KNOWN_BUGS#L161-L168
> [2]: https://github.com/curl/curl/commit/4acc9d3
> [3]: https://github.com/curl/curl/commit/f8be737
>
>
> The additional data appears to be separately compressed gzip data that we are treating as an unexpected trailer. I'm not sure whether or not that's allowed. I opened an issue for it [1], please follow up there if you have any additional information.

  Thanks, I've added a comment to the issue.

  Dmitri

>
> [1]: https://github.com/curl/curl/issues/3796
>
> -------------------------------------------------------------------
> Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library
> Etiquette: https://curl.haxx.se/mail/etiquette.html
-------------------------------------------------------------------
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library
Etiquette: https://curl.haxx.se/mail/etiquette.html
Received on 2019-04-21