>
>
> On Mon, 6 Jun 2011, Jimish Shah wrote:
>
> > I am trying to use libcurl to scale to several thousand https connections
> to
> > an external web server.
>
> Use the multi_socket API (and something like libev or libevent) and build
> libcurl to use c-ares for resolving. We've had users do over 60K
> connections
> on a single host.
>
> > If i use multi-handle and add say 1000 easy handles to it. If each of the
> > thousand easy handles need to connect simultaneously , i noticed that
> there
> > is a blocking "select ()" during the connect phase as explained below.
>
> That's not quite accurate. See below.
>
> > Curl_multi_perform() --> multi_runsingle() --> Curl_connect() ....... -->
> > Singleipconnect --> waitconnect() --> Curl_socket_ready()* --> select ()*
> > every one second until error or success or timeout
>
> Curl_connecthost() calls singleipconnect() on lib/connect.c:1084 (in
> current
> git)
>
> There you can see how it changes the timeout argument to 0 (zero) if the
> multi
> interface is used. Therefore, within the singleipconnect() function where
> it
> calls waitconnect() the timeout argument is zero and thus it will not wait,
> it
> will just check the status of the socket instantly and then move on.
>
> If you can see any other flow, please tell me.
>
> --
>
> / daniel.haxx.se
>

Thanks Daniel. You are right, singleipconnect does pass a 0 timeout. Sorry i
missed it. So now my question is as follows

Curl_multi_perfom() --> multi_runsingle () for every easy handle.
Now singipconnect() is invoked with 0 timeout, connect() returns -1 and
errno set to EINPROGRESS (maybe for a slow web server)

/lib/connect.c --> singleipconnect()

  /* Connect TCP sockets, bind UDP */
  if(!isconnected && (conn->socktype == SOCK_STREAM)) {
    rc = connect(sockfd, &addr.sa_addr, addr.addrlen);
<<<<clip>>>>
    case EINPROGRESS:
<<<<clip>>>>
      rc = waitconnect(conn, sockfd, timeout_ms);
<<<<clip>>>>
  if((WAITCONN_TIMEOUT == rc) &&
     (data->state.used_interface == Curl_if_multi)) {
    *sockp = sockfd;
    return CURLE_OK;
  }

So sockfd is set to the actual socket and we return. I cant figure out who
keeps track of this socket if it returns here. Coz at the top level,
multi_runsingle is invoked for each easy handle until it succeeds or fails.
See extracted code below.

/lib/multi.c --> Curl_multi_perform()

    do
      result = multi_runsingle(multi, now, easy);
    while (CURLM_CALL_MULTI_PERFORM == result);

As soon as Curl_multi_perform is done with this loop, we call fd_set which
expects all the fd's to be connected.

So in simple terms, if the socket returns immediately, who keeps track and
who triggers when the socket is connected.

Is it mandatory to use libevent to handle the same ?

Sorry for the long email. Pls correct me if my understanding is incorrect.

Thanks
Jimish

-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-library
Etiquette: http://curl.haxx.se/mail/etiquette.html
Received on 2011-06-07