curl / Mailing Lists / curl-users / Single Mail


Need some help about large data from web

From: <>
Date: Thu, 10 Nov 2016 13:12:26 +0800

Hi all
I have a C program which run to get the data from a web API.
And when the web return a large json data, the curl can’t deal with it.
Every time the program return (core dump) error, and I want to know how to deal with the indefinite length data use the libcurl use C.
And my code like this:
size_t write_data(void *ptr, long unsigned int size, long unsigned int nmemb, void *stream)

    strcat(stream, (char *)ptr);
    return size*nmemb;

char *down_file(char *url_address)
    static unsigned char str[64000] = {""};
    char *p2s;

     /* set the download address */
    curl_easy_setopt(curl_handle, CURLOPT_URL, url_address);
     /* set the timeout */
    curl_easy_setopt(curl_handle, CURLOPT_TIMEOUT, 100);
    /* set the writedata function */
    curl_easy_setopt(curl_handle, CURLOPT_WRITEFUNCTION, write_data);
    /* set the wirte variable */
    curl_easy_setopt(curl_handle, CURLOPT_WRITEDATA, str);
    /* set the user-agent field */
    curl_easy_setopt(curl_handle, CURLOPT_USERAGENT, "libcurl-agent/1.0");

    /* download */
    res = curl_easy_perform(curl_handle);
    //str[MAX_LINE-1] = "\0";
    if(res != CURLE_OK)
        fprintf(stderr, "curl_easy_perform() failed: %s\n", curl_easy_strerror(res));
        return NULL; //judge the download if successful
    printf("%lu bytes retrieved\n", strlen(str));
    return str; }


List admin:
Received on 2016-11-10