cURL / Mailing Lists / curl-library / Single Mail

curl-library

RE: Unable to retrieve some document header

From: Yanick Pelletier <ypelletier_at_copernic.com>
Date: Mon, 15 Oct 2001 17:41:09 -0400

[ what to do when HEAD doesn't work ]

Ok, i agree with you with the fact that curl should not add layer to fix
bugs from servers.

First of all i work on Windows 2000.

I have try your solution but i have un bug with it. Here the code that cause
the bug:

    m_CURL = curl_easy_init();
    curl_easy_setopt(m_CURL, CURLOPT_URL, "http://www.copernic.com");
    curl_easy_setopt(m_CURL, CURLOPT_FILETIME, 1);
    curl_easy_setopt(m_CURL, CURLOPT_FOLLOWLOCATION, 1);
    curl_easy_setopt(m_CURL, CURLOPT_WRITEHEADER, 0);
    curl_easy_setopt(m_CURL, CURLOPT_HEADERFUNCTION, HeaderParsingFunc);
    curl_easy_setopt(m_CURL, CURLOPT_FILE, 0);
    curl_easy_setopt(m_CURL, CURLOPT_WRITEFUNCTION, DataWriteFunc);
    res = curl_easy_perform(m_CURL);

    curl_easy_setopt(m_CURL, CURLOPT_URL, GetURI().GetURI().c_str());
    curl_easy_setopt(m_CURL, CURLOPT_FILE, 1);
    curl_easy_setopt(m_CURL, CURLOPT_HEADERFUNCTION, 0);
    res = curl_easy_perform(m_CURL);
    
    curl_easy_cleanup(m_CURL);

Here the code of the two function used to write the data:

        size_t DataWriteFunc(void *pi_pBuffer, size_t pi_Size, size_t
pi_NbBytes, void *po_pFile)
        {
            if (!po_pFile)
                return -1;
                return pi_Size * pi_NbBytes;
        }

        size_t HeaderParsingFunc(void *pi_pBuffer, size_t pi_Size, size_t
pi_NbBytes, void *po_pFile)
        {
                return pi_Size * pi_NbBytes;
        }

With this code the first call to curl_easy_perform() will always abort when
retrieving the body data, so we only recieve the header. On the second call
to curl_easy_perform() we don't retreive the header so we only receive the
body.

When the above code is executed in it step-by-step in debug mode it run
fine, but when i just let it run free it loop forever (in fact it loop a
long time, a verry long time!!!) in Transfert(). What i have found so far
its that the second request perform verry baddly if i don't "eat" all the
data generated by the first call to curl_easy_cleanup().

Does any body have a solution that i can try, except the solution to process
all the data of the first call!

Thanks!
Yanick

-----Original Message-----
From: Daniel Stenberg [mailto:daniel_at_haxx.se]
Sent: Monday, October 15, 2001 9:22 AM
To: Yanick Pelletier
Cc: libcurl Mailing list
Subject: RE: Unable to retrieve some document header

On Mon, 15 Oct 2001, Yanick Pelletier wrote:

[ what to do when HEAD doesn't work ]

> Hum. Your solution work since i have already test it, but my biggest
> problem it to ask the server for a document that a don't really want,
> it for this reason that the HEAD command exist. So does it is possible
> for curl to check the result of the HEAD command to see if an header
> is returned, if no header is found it can send the GET command to
> retrieve it.

Yes, curl *could* probably be patched into doing this.

But why should curl do this? I don't think curl should or even could (in a
good enough way) add a layer that makes it fix blatant server errors for the
applications.

Why don't you just try NOBODY first and if that doesn't return any header,
you try without it?

-- 
    Daniel Stenberg -- curl groks URLs -- http://curl.haxx.se/
Received on 2001-10-15