On Sun, 6 Nov 2011, Yehezkel Horowitz wrote:

> This is my fist post to this list, so I want to use this opportunity to
> thanks all of you (mainly Daniel) for the development, maintenance and
> support efforts you invest on this amazing tool "cUrl".

Thanks for these friendly words and welcome to the project!

> I want to use curl_multi interface along with epoll (can't use select since
> I want to perform much more than 1024 requests in parallel).
>
> My problem is that I can't get the "new" sockets that I need to add to the
> epoll_ctl, just the whole set of fds (by using curl_multi_fdset).

Right, the original multi interface API was never made for event based systems
but was instead perhaps a bit naively designed to rely on select().

You need to switch over to the curl_multi_socket_action() paradigm to get the
event based style going. http://curl.haxx.se/libcurl/c/hiperfifo.html is an
example using it. That API tells your application about each file descriptor
to wait for and you then tell libcurl for each activity on one of the
descriptors that belong to libcurl.

> I tried to overcome this problem my managing two "multi" handle:

I don't think that's a very good idea.

> How should I proceed? Is there an option to get my goal (use more than 1024
> easy handles in parallel with curl_multi interface)?
>
> Take into account - my goal is to have ~20K connections opened in parallel.

While I developed the multi_socket API I regularly used 9K simultaneous
connections and we have friends that have gone past 60K connections using a
single multi handle.

-- 
  / daniel.haxx.se
-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-library
Etiquette:  http://curl.haxx.se/mail/etiquette.html