On 6/2/2010 1:47 PM, Daniel Stenberg wrote:
> So how many easy handles do you add to the multi handle?

Well, perhaps my whole design is wrong. What I do now involves no limits
on the number of easy handles and no pre-allocation of handles as is
done in the 10-at-a-time.c example. What I do is, whenever a transfer
(upload) is ready to go I check the multi handle to see if there's an
"idle" easy handle. If so I use it, if not I create a new handle and add
it to the stack. Thus, when doing 407 simultaneous transfers I may have
407 easy handles on the multi stack, unless some of the early ones
finish before the last ones go on. I've pasted the relevant code below.
It works fine except when this boundary condition is reached.

> And btw, what OS are you doing this on?

Solaris 10 and Linux (CentOS 5).

>> I see there is a CURLMOPT_MAXCONNECTS setting but this seems to be a
>> little different. Does anyone have example code showing how to limit
>> the number of active transfers to 10 or 100 or so?
>
> You control that by not adding more handles than you want to be
> transfered in parallell. CURLMOPT_MAXCONNECTS is then used to make sure
> than no more than the specified amount is kept open.

Here's what I do now (snipped and simplified a bit, may not compile):

/// Look through the multi-handle for an easy handle which
/// has finished its transfer and is ready to be used again. If no
/// free handle is found, a new one is generated. This function
/// is guaranteed to return a usable handle in either case.
/// @return a curl easy handle ready for use
CURL *
http_async_get_free_curl_handle(void)
{
     CURLMsg *curlmsg;
     int msgcnt = 0;
     CURL *curl = NULL;
     static int in_use;

     while (!curl && (curlmsg = curl_multi_info_read(MultiHandle,
&msgcnt))) {
        if (curlmsg->msg == CURLMSG_DONE) {
            int rc;

            curl = curlmsg->easy_handle;

            rc = http_check_connect_result(curl, curlmsg->data.result);
            if (rc && prop_is_true(P_STRICT_UPLOAD)) {
                die("unable to get libcurl handle (%d)", rc);
            }

            curl_multi_remove_handle(MultiHandle, curl);
        }
     }

     if (curl) {
        printf("Reusing handle %p", curl);
     } else {
        curl = curl_easy_init();
        in_use++;
        printf("New handle %p (in use=%d)", curl, in_use);
     }

     return curl;
}

// Internal service routine.
static int
http_check_connect_result(CURL *curl, CURLcode res)
{
     int rc = 0;
     long httpcode = -1;

     curl_easy_getinfo(curl, CURLINFO_RESPONSE_CODE, &httpcode);
     if (res != CURLE_OK || httpcode != 200) {
        // print error message
        rc = 2;
     }

     return rc;
}

-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-library
Etiquette: http://curl.haxx.se/mail/etiquette.html
Received on 2010-06-03