Daniel Stenberg wrote:
> If we have 10000 active download connections (having added 10000 easy
> handles to a single multi handle), then we can in fact get traffic on one
> or more of all those connections.

Yes. Although, unless you have a very fast network compared with the
CPU, only a few connections will be ready at each select/poll.

> I don't see how libcurl can do anything but to return the 10000
> descriptors to the app saying "here, wait for this action on one or
> more of these".

That's fine, it's a question of _how_ you return those 10000 descriptors.

If you return an array of 10000 entries which the app has to scan,
that's not good.

If you instead return an array containing only the differences since
the previous call (as an array of add/remove/change operations), that
is good because the app (and kernel) only have to scan the
differences.

That's not as complicated as it sounds.

After the app calls select() or whatever, and then passes curl the 20
ready descriptors to process, _only_ those 20 descriptors are going to
cause differences before the next return to the app. (Ok, a few more
due to timeouts, but only a few). So you just build the array to
return based on each descriptor that is actually processed during that
call into curl.

Actually, I think the callback function which is used to inform the
app about changes in the interest set is nicer than building an array.
But they are algorithmically similar - and both much faster than
passing the whole big array of mostly unchanging fd numbers.

-- Jamie
Received on 2005-01-31