On May 19, 2014, at 2:46 PM, Madhurima Das <madhurima.das_at_gmail.com> wrote:

> I am a beginner in both C programming and libcurl and writing a program to fetch 1000 data values from a website. The website provides a job number and is redirected into another page for the results. Since, the code I have written is almost 500 lines, I am giving a general flow of the program and a short code which I think is the problematic area:

You should use the multi interface, it will automatically open a limited number of concurrent connections, correctly reuse them, etc. If you make a 1000 requests at once, I’m not surprised that your network, the server you are talking to or something else is overwhelmed and causes request failures.

http://curl.haxx.se/libcurl/c/libcurl-multi.html

Regards,
Fabian
-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-library
Etiquette: http://curl.haxx.se/mail/etiquette.html
Received on 2014-05-20