curl-library
Saving data received from http server
Date: Fri, 12 Nov 2010 17:28:57 +0530
WANTED: save the incoming data from http server.
WHAT I GOT: It works
PROBLEM: A dirty brute-force approach used to save data. Have
explained the problem in comments inside the program. Anyone has
better idea ?
I have written a simple program to receieve data from any HTTP server
(learning to use libcurl in the compnay where I work). I can get the
data easily using this small program written by me. It outputs the
data to stdout (default in libcurl). (VERSION 1). For saving data I
wrote VERSION 2. comment/uncomment to test both compilable & running
versions.
#include <stdio.h>
#include <curl/curl.h>
size_t save_data(void* buffer, size_t size, size_t nmemb, void* incoming_data);
enum {
SIZE_ERROR_ARR = CURL_ERROR_SIZE * 4
};
int main(void)
{
CURL* curl;
CURLcode res;
char error_arr[SIZE_ERROR_ARR+1] = {0};
curl = curl_easy_init();
if(curl)
{
/* Use one VERSION by commenting the other */
/* VERSION 1 : Outputs to stdout */
curl_easy_setopt(curl, CURLOPT_URL, "http://google.com");
res = curl_easy_perform(curl);
/* VERSION 2 : Saves to array
curl_easy_setopt(curl, CURLOPT_URL, "http://google.com");
curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, save_data);
curl_easy_setopt(curl, CURLOPT_ERRORBUFFER, error_arr);
res = curl_easy_perform(curl);
*/
if(res)
{
printf("IN: %s @%d: Some error occured while curl_easy_perform(),
ERRORBUFFER said: \n", __FILE__, __LINE__);
printf("%s\n", error_arr);
}
/* always cleanup */
curl_easy_cleanup(curl);
}
return 0;
}
size_t save_data(void* buffer, size_t size, size_t nmemb, void* incoming_data)
{
size_t idx;
const size_t END_INDEX = size * nmemb;
char* src = (char*) buffer;
char dest[100001] = {0};
/* How to know the size of array in advance.
char dest[END_INDEX + 1] = {0}; will not work as variable-size
arrays are not allowed in ANSI standard. But using variable-size
is good solution to the problem ???
malloc() will be expensive for 10 million http sends/receive
waitings to happen.
So what to do ???
*/
for(idx = 0; idx < END_INDEX; ++idx)
{
dest[idx] = *src++;
}
printf("--------------------------------------\n");
/* I wonder what 4th argument is used for when it contains garbage :-o */
printf("incoming_data = %s\n", (char*)incoming_data);
printf("dest = %s\n", dest);
printf("--------------------------------------\n");
return END_INDEX;
}
================ OUTPUT ========================
[arnuld_at_dune Module-cURL]$ gcc -ansi -pedantic -Wall -Wextra
curl-simple.c -lcurl
[arnuld_at_dune Module-cURL]$ ./a.out
<HTML><HEAD><meta http-equiv="content-type" content="text/html;charset=utf-8">
<TITLE>301 Moved</TITLE></HEAD><BODY>
<H1>301 Moved</H1>
The document has moved
<A HREF="http://www.google.com/">here</A>.
</BODY></HTML>
[arnuld_at_dune Module-cURL]$
-- http://uttre.wordpress.com/2008/05/14/the-lost-love-of-mine/ ------------------------------------------------------------------- List admin: http://cool.haxx.se/list/listinfo/curl-library Etiquette: http://curl.haxx.se/mail/etiquette.htmlReceived on 2010-11-12