cURL / Mailing Lists / curl-library / Single Mail

curl-library

get several pages instead of one with cookies

From: Ilann Cohen <ilann_at_ilann.net>
Date: Sat, 11 Dec 2004 12:02:49 +0100

Hi,

[curl/libcurl v7.11.1 on a Windows i686 with Cygwin system]

With libcurl, I’m trying to post parameters to a form, and get the
result page per page. A cookie is provided by the server while I’m
getting result page 1, and I send it back to the server for subsequent
requests.
It works fine, except that when I fetch page 2, I get pages 1 to 2, when
I fetch page 3, I get pages 1 to 3 etc. I mean that libcurl returns a
buffer containing all previous pages, concatenated to each other !!

Here is what I do :

INIT (called once) :
    struct MemoryStruct *chunk;
    CURL *curl_handle;
    
    chunk = &g_mychunk; /* GLOBAL variable */
    chunk->memory=NULL; /* we expect realloc(NULL, size) to work */
    chunk->size = 0; /* no data at this point */
    
    /* init the curl session */
    curl_handle = curl_easy_init();
    
    /* specify URL to get */
    curl_easy_setopt(curl_handle, CURLOPT_COOKIEFILE, "nofilename.txt");
    
    /* send all data to this function */
    curl_easy_setopt(curl_handle, CURLOPT_WRITEFUNCTION,
url_writeMemoryCallback);
    
    /* we pass our 'chunk' struct to the callback function */
    curl_easy_setopt(curl_handle, CURLOPT_FILE, (void *)chunk);

GET PAGE (called for every page):

    /* specify URL to get */
    curl_easy_setopt(curl_handle, CURLOPT_URL, url_name);
   
    /* get it! */
    curl_easy_perform(curl_handle);

CLEAN (called once)
    /* cleanup curl stuff */
    curl_easy_cleanup(curl_handle);

Note: the url of page 1 is:
http://www.explorimmo.com/ListeAnnonce.shtml?MEMO_CRITERE=0&Recherche.ty
peTransaction=VENTE&Recherche.devise=euros&Recherche.typeBien=_APPARTEME
NT%3B_DUPLEX%3B_CHAMBRE%3B_LOFT%3B_ATELIER%3B&Recherche.Anciennete=NULL&
Recherche.nbPieces.min=1&Recherche.nbPieces.max=2&Recherche.surface.min=
10&Recherche.surface.max=30&Recherche.prix.min=10000&Recherche.prix.max=
150000&Recherche.pays=FRA&regionMer=Indiff%E9rent&regionMontagne=Indiff%
E9rent&Recherche.localisation=75011&Recherche.localisation_temp=&Recherc
he.date=3%2F12%2F2004&Recherche.professionnels=checked&Recherche.particu
liers=checked&Submit.x=34&Submit.y=4

The url for pages 2 to n is (LIST_PAGE set to page number):
http://www.explorimmo.com/ListeAnnonce.shtml?ROWS=10&CODE_TRI=DATE_MODIF
&UNIVERS=null&Recherche.decoupe=2&Recherche.typeTransaction=VENTE&Recher
che.typeBien=_APPARTEMENT;_DUPLEX;_CHAMBRE;_LOFT;_ATELIER;&Recherche.Anc
iennete=NULL&Recherche.nbPieces.min=1&Recherche.nbPieces.max=2&Recherche
.surface.min=10&Recherche.surface.max=30&Recherche.prix.min=10000&Recher
che.prix.max=150000&Recherche.meuble=NULL&Recherche.localisation=75011&R
echerche.pays=FRA&paramPub=1&LISTE_PAGE=2

What's going on when you call curl_easy_setopt(curl_handle, CURLOPT_URL,
url_name) several times with the same handle ? Do I have to clean
something before setting a new url ?

Thanks,
Ilann
Received on 2004-12-11