cURL / Mailing Lists / curl-library / Single Mail

curl-library

RE: Lost data in chunk HTTP response using libcurl

From: Yehezkel Horowitz <horowity_at_checkpoint.com>
Date: Wed, 17 Oct 2012 16:23:03 +0200

>I'm using libcurl to build a HTTP proxy in C++ currently. It worked perfect for non-chunk HTTP(those HTTP responses with "Content-length"). However, when it needed to handle chunk HTTP, the responses always lost data(checked through byte size and md5sum) compared with the responses I captured using wireshark. The most obviously losing data are the chunked sizes at the beginning of the HTTP responses' message body and the digital 0 to indicate ending at the end.

By default, libcurl will decode the chunks for you. So your write callback will see only the "real" data.
In case you are acting as a proxy and pass this data you should either remove the "Transfer-Encoding: chunked" header (in your write_header callback) OR kindly ask libcurl not to decode the stream with CURLOPT_HTTP_TRANSFER_DECODING option.

HTH

Yehezkel Horowitz

-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-library
Etiquette: http://curl.haxx.se/mail/etiquette.html
Received on 2012-10-17