On Mon, 19 May 2008, Emil Romanus wrote:

> I dug around in the code today looking for some place where we could
> introduce a wake up call for waiting easy handles. So far, the fault
> when it comes to my problem has the following path:
> * 5 handles are added, each pointing to the same host
> * DNS lookup is started by the first one
> * The 4 others are put in a WAITDO state (not sure, but so it seems)
> * DNS lookup completes, first handle continues and completes transfer

I assume you're now talking 7.18.1 with pipelining enabled? I'm not currently
testing that case since I have my described problems already without
pipelining and getting them sorted first feels like a good idea to me.

> Presumably, we are only waiting for one single socket during the DNS
> lookup.

Only if you do this with pipelining.

> Once this socket needs attention, curl_multi_socket() is called on that
> socket. Here, the problem arises, since the 4 other handles are never
> notified of the DNS lookup being done.

Well that's of course not fine behavior. When a reqeuest is done sending a
pipelined request, it should make the next sender "in queue" get properly
adjusted to change state from WAITDO to DO and thus send off its request.

> If curl_multi_socket_all() is never called, or curl_multi_socket() isn't
> called due to timeout, then attention will *never* be given to the 4 waiting
> handles.

No, that logic doesn't hold. The case when you call curl_multi_socket() due to
a timeout is exactly the same as when you call curl_multi_socket() due to a
socket action, with the only difference that there isn't the primary socket to
check. All calls to the curl_multi_socket*() functions will check the timer
tree for handles that have timed out and need action done.

> However, the bottleneck of calling curl_multi_socket_all() instead of
> curl_multi_socket() on every time attention is needed by a socket is notably
> big.

... which is we have made libcurl to basically never require
curl_multi_socket_all().

> If libcurl could somehow change the state and update the sockets (through
> the callback, I mean) to every (host-wise) related easy handle in a multi
> handle when DNS lookup completes. Though I'm not able to patch this myself,
> since I lack enough knowledge about the multi stack, especially how I would
> find out which easy handles that are waiting for the host lookup to
> complete.

They're added to linked lists (one for sendind and one for receiving). See
addHandleToSendOrPendPipeline() for send_pipe and recv_pipe stuff.

-- 
  / daniel.haxx.se