Hi everybody.
I use libcurl to get very many html-pages via multi interface. And the problem is that when I add to CURLM* handle a lot of easy handles (for example, some thousands), or create many threads to divide it and accelerate my program, for most of them (about 70-80 %) libcurl returns error "Couldn't resolve host name".

But when I select some URLs from the list with this error (for example 10-15 URLs) and check their via browser or curl (command line) or libcurl via my program - all of them are available and no errors occurs! I can load pages successfully!

My code like this approximately:
CURLM *mc;
//initialize multi handle,
//initialize easy handles with `curl_easy_setopt`

//adding it to multi interface with `curl_multi_add_handle`

CURLMsg *msg = 0;
int in_process = -1;
int msg_queue = 0, nums = 0;

while ( in_process )
{
      curl_multi_perform(mc, &in_process);

      if ( in_process )
      {
          int nums;
          curl_multi_wait(mc, 0, 0, 500, &nums);
      }

      while ( (msg = curl_multi_info_read(mc, &msg_queue)) )
      {
          //process message
      }
}

//cleanup

libcurl 7.28.1, CentOS 5.9 x86_64 and CentOS 6.0 i686.

Please, can anybody help me with this problem? Thanks in advance.

-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-library
Etiquette: http://curl.haxx.se/mail/etiquette.html
Received on 2013-02-23