cURL / Mailing Lists / curl-library / Single Mail

curl-library

Re: Problem with curl_multi C API

From: Etienne Bougoin <etienne.bougoin_at_c-s.fr>
Date: Thu, 18 Mar 2010 14:51:40 +0100

Le 18 mars 10 à 14:33, Etienne Bougoin a écrit :

> Hello,
>
> I have a problem with curl multi API on several platforms.
>
> I try to fetch map tiles from open street map and I use curl_multi
> API to fetch them faster.
>
> My problem is when one or more request return 404 error, the other
> request doen't work and images are not downloaded correctly.
>
> I use this code to generate easy_handles:
>
> // Initialize curl handle
> CURL * lEasyHandle;
> lEasyHandle = curl_easy_init();
>
> if (!lEasyHandle)
> {
> itkExceptionMacro( <<"Tile Map IO : Curl easy handle init
> error.");
> }
>
> // Create file
> FILE* lOutputFile = fopen(tileInfo.filename.c_str(), "w");
>
> if (lOutputFile == NULL)
> {
> itkExceptionMacro(<<"TileMap read : bad file name.");
> }
>
> // Add file to vector
> m_ListFiles.push_back(lOutputFile);
>
> // Param easy handle
> curl_easy_setopt(lEasyHandle, CURLOPT_USERAGENT, m_Browser.data());
> curl_easy_setopt(lEasyHandle, CURLOPT_URL,
> m_ListURLs.back().data());
> curl_easy_setopt(lEasyHandle, CURLOPT_WRITEDATA,
> m_ListFiles.back());
>
> // Add easy handle to multi handle
> curl_multi_add_handle(m_MultiHandle, lEasyHandle);
>
> // Add hanle to vector
> m_ListCurlHandles.push_back(lEasyHandle);
>
> And this one to perform requests:
>
> // Configure multi handle - set the maximum connections
> curl_multi_setopt(m_MultiHandle, CURLMOPT_MAXCONNECTS, 10);
> curl_multi_setopt(m_MultiHandle, CURLMOPT_PIPELINING, 0);
>
> // Perform
> int lStillRunning;
>
> while(
> CURLM_CALL_MULTI_PERFORM == curl_multi_perform(m_MultiHandle,
> &lStillRunning)
> );
>
> // Now get that URL
> while(lStillRunning)
> {
> struct timeval timeout;
> int rc; // Return code
>
> fd_set fdread;
> fd_set fdwrite;
> fd_set fdexcep;
> int maxfd;
>
> FD_ZERO(&fdread);
> FD_ZERO(&fdwrite);
> FD_ZERO(&fdexcep);
>
> /* set a suitable timeout to play around with */
> timeout.tv_sec = 0;
> timeout.tv_usec = 1;
>
> /* get file descriptors from the transfers */
> curl_multi_fdset(m_MultiHandle, &fdread, &fdwrite, &fdexcep,
> &maxfd);
>
> rc = select(maxfd+1, &fdread, &fdwrite, &fdexcep, &timeout);
>
> switch(rc) {
> case -1:
> /* select error */
> break;
> case 0:
> /* timeout */
> default:
> /* timeout or readable/writable sockets */
> while(
> CURLM_CALL_MULTI_PERFORM ==
> curl_multi_perform(m_MultiHandle, &lStillRunning)
> );
> break;
> }
> }
>
> int remaining_msgs = 1;
> int error = 0;
> CURLMsg *msg;
> while (remaining_msgs)
> {
> msg = curl_multi_info_read(m_MultiHandle, &remaining_msgs);
> if (msg != NULL)
> {
> if (CURLE_OK != msg->data.result)
> error = 1;
> }
> }
>
> Can you help me to find where I do something wrong?
>
> Many thanks
>
> Etienne
> -------------------------------------------------------------------
> List admin: http://cool.haxx.se/list/listinfo/curl-library
> Etiquette: http://curl.haxx.se/mail/etiquette.html

I forgot to specify on wich platforms I make the test:

- Mac OSX 10.5.8
          - curl v7.16.3
        - curl v7.19.4
        - curl v7.20.0

- CENT OS
        - curl v7.15.5

- Debian Lenny
        - curl v7.18.2

But it works fine on this platform:

- Ubuntu v8.04
        - curl v7.18.0

Etienne

-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-library
Etiquette: http://curl.haxx.se/mail/etiquette.html
Received on 2010-03-18