Jamie Lokier wrote:

> That's fine, it's a question of _how_ you return those 10000 descriptors.
>
> If you return an array of 10000 entries which the app has to scan,
> that's not good.
>
> If you instead return an array containing only the differences since
> the previous call (as an array of add/remove/change operations), that
> is good because the app (and kernel) only have to scan the
> differences.

So, you're suggesting some async notification instead of select/poll?

In that case, curl could just provide a callback every time it added or
deleted a socket, and your application could deal with everything on it's
own. Curl would just need a 'go process this socket fd' method after your
app has determined something needs to be done to the socket.

That said, select/poll is not so bad as you may think, and if you have 10k
sockets serviced by a single application, you may instead want to consider
breaking your own app into multiple processes.

> After the app calls select() or whatever, and then passes curl the 20
> ready descriptors to process, _only_ those 20 descriptors are going to
> cause differences before the next return to the app. (Ok, a few more
> due to timeouts, but only a few). So you just build the array to
> return based on each descriptor that is actually processed during that
> call into curl.

If your app is still calling select or poll, you've gained almost exactly
nothing because the kernel is still going to go through all 10k FDs in O(N)
time.

Ben

-- 
Ben Greear <greearb_at_candelatech.com>
Candela Technologies Inc  http://www.candelatech.com