curl-users
Need some help about large data from web
Date: Thu, 10 Nov 2016 13:12:26 +0800
Hi all
I have a C program which run to get the data from a web API.
And when the web return a large json data, the curl can’t deal with it.
Every time the program return (core dump) error, and I want to know how to deal with the indefinite length data use the libcurl use C.
And my code like this:
size_t write_data(void *ptr, long unsigned int size, long unsigned int nmemb, void *stream)
{
strcat(stream, (char *)ptr);
return size*nmemb;
}
char *down_file(char *url_address)
{
static unsigned char str[64000] = {""};
char *p2s;
/* set the download address */
curl_easy_setopt(curl_handle, CURLOPT_URL, url_address);
/* set the timeout */
curl_easy_setopt(curl_handle, CURLOPT_TIMEOUT, 100);
/* set the writedata function */
curl_easy_setopt(curl_handle, CURLOPT_WRITEFUNCTION, write_data);
/* set the wirte variable */
curl_easy_setopt(curl_handle, CURLOPT_WRITEDATA, str);
/* set the user-agent field */
curl_easy_setopt(curl_handle, CURLOPT_USERAGENT, "libcurl-agent/1.0");
/* download */
res = curl_easy_perform(curl_handle);
//str[MAX_LINE-1] = "\0";
if(res != CURLE_OK)
{
fprintf(stderr, "curl_easy_perform() failed: %s\n", curl_easy_strerror(res));
return NULL; //judge the download if successful
}
else
{
printf("%lu bytes retrieved\n", strlen(str));
}
return str;
}
THX
Jay
-------------------------------------------------------------------
List admin: https://cool.haxx.se/list/listinfo/curl-users
FAQ: https://curl.haxx.se/docs/faq.html
Etiquette: https://curl.haxx.se/mail/etiquette.html
Received on 2016-11-10