cURL / Mailing Lists / curl-library / Single Mail

curl-library

Re: Decompression problems

From: Marco Toldo <marcoxtoldo_at_gmail.com>
Date: Fri, 17 Apr 2009 18:20:24 +0200

Hi,

possibly it's an encoding problem.

When I began using zlib and deflate compression format I was never
able to decompress data sent from a server (I always got "header
check" errors too).
Later I discovered that compressed data are usually base64 encoded.
For this reason a frequent compression/decompression scheme is the following:

deflate ----> encode Base 64 ----> Transmission ----> decode Base
64 ---> enflate

Base64 encoding is described here: http://www.faqs.org/rfcs/rfc3548.html

JDK from Sun has built-in classes to encode/decode base64 in Java.
As I needed them for C/C++ (which doesn't provide them) I wrote them on my own.

If you need them feel free to ask me.

Regards,

Marco

2009/4/17, Jonathan Wallace <jwallace_at_livetechnology.com>:
> I hope someone can help me figure out what's going wrong with this
> thing, I've been fighting with it for months. I have a bit of software
> that is supposed to pull in compressed data from a server. The server
> uses zlib to do compression, the exact same version that I have (and
> I've tried multiple dlls, they all don't work, so that leads me to
> believe it's a libcurl problem. I must be doing something wrong, but so
> far no one has been able to figure out what.
>
> I've used Fiddler to see what's coming back, it is correctly compressed
> data (deflate,) so it's not a communication problem.
>
> The error buffer has this in it: Error while processing content
> unencoding: incorrect header check
>
> Here is my source code:
>
> string httpLoader::DownloadFile(string & xurl, bool login){
> /*
> This method requires that the server address be including in the
> xurl, if you are
> using the server which has been connected to in the login process,
> use DownloadFileMyServer
> */
> wxString temp = wxString::FromAscii(xurl.c_str());
> temp.Replace(_T("\\\\\\\\"), _T("\\\\"), true);
> xurl = string(temp.mb_str());
>
> wxSetCursor(wxCURSOR_ARROWWAIT);
> //initialize the http component
> CURLcode result;
> if (curl){
> char errorBuffer[CURL_ERROR_SIZE];
> string buffer;
> curl_easy_setopt(curl, CURLOPT_ERRORBUFFER, errorBuffer);
> curl_easy_setopt(curl, CURLOPT_POST, 0);
> curl_easy_setopt(curl, CURLOPT_URL, xurl.c_str());
> SetOptions(true);
> curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, writer);
> curl_easy_setopt(curl, CURLOPT_WRITEDATA, &buffer);
>
> result = curl_easy_perform(curl);
>
> if(result == CURLE_OK){
> return buffer;
> } else {
> Debug::Log("result != CURLE_OK");
> Debug::Log(errorBuffer);
> }
>
> } else {
> Debug::Log("Unable to create CURL Instance");
> }
> return "";
> }
>
> void httpLoader::SetOptions(bool login){
> if (XRequestID.empty()){ //create the XRequestID
> wxDateTime now = wxDateTime::Now();
> wxString nowtime = now.Format(_T("%m%d%y_%Hh%M"));
> XRequestID = "x-requestid: LiveRuntime" + string(nowtime.mb_str());
> }
>
> if (DebugMode & LR_DEBUG_FIDDLER) curl_easy_setopt(curl,
> CURLOPT_PROXY, "127.0.0.1:8888");
> if (DebugMode & LR_DEBUG_HTTPLOG){
> curl_easy_setopt(curl, CURLOPT_DEBUGFUNCTION, my_trace);
> curl_easy_setopt(curl, CURLOPT_VERBOSE, 1L);
> }
>
> struct curl_slist *headers = NULL;
> headers = curl_slist_append(headers, "Accept: */*");
> if (DebugMode & LR_DEBUG_COMPRESSION){
> headers = curl_slist_append(headers, "Accept-Encoding: deflate");
> curl_easy_setopt(curl, CURLOPT_ENCODING, "deflate");
> }
> if (!LOSID.empty()){
> string s = "LOSID=" + LOSID;
> headers = curl_slist_append(headers, s.c_str());
> }
> headers = curl_slist_append(headers, "Accept-Language: en-us");
> headers = curl_slist_append(headers, "User-Agent: LiveRuntime");
> headers = curl_slist_append(headers, XRequestID.c_str());
> headers = curl_slist_append(headers, "Connection: Keep-Alive");
> headers = curl_slist_append(headers, "Content-Type:");
> headers = curl_slist_append(headers, "Authorization:");
> curl_easy_setopt(curl, CURLOPT_HTTPHEADER, headers);
> }
>
Received on 2009-04-17