Daniel Stenberg wrote:
> I don't mind callbacks in general, I just don't see them fit into this very
> comfortably.
>
> I think of an application's main loop looking like this:
>
> while(still_things_to_do) {
> curl_multi_sock(args...); /* get file descriptors */
> set_things_up(); /* prepare for poll/select/similar */
> select() or poll() or similar(); /* waits on 10000 connections */
> curl_multi_perform();
> }
>
> Of course curl_multi_sock() could be made to call a callback for what file
> descriptors that are added/removed from the previous call. If that is what
> people in general prefer.
>
> If we're talking details, how do you suggest the specific function
> prototype(s) would be? And how can we adjust the API and the simple example
> above to as good as possible be prepared for a huge amount of connections?

There are 4 functions to operate on an abstract fdset:

    fdset_init - initialise the app-private fdset data structure
    fdset_modify - register or change interest in a single fd
    fdset_wait - call select() or poll() or something_else()
    fdset_destroy - destroy the app-private fdset data structure

Signatures:

    void * fdset_init (void);
    void * fdset_modify (void * fdset, void * handle, int fd, int what,
                         void (*callback) (void *), void * arg);
    int fdset_wait (void * fdset, struct timeval * timeout);
    void fdset_destroy (void * fdset);
  
The function which needs most explanation is fdset_modify:

   void * fdset_modify (void * fdset, void * handle, int fd, int what,
                        void (*callback) (void *, int), void * arg);

The "what" argument has one of five values:

   0 CURL_POLL_NONE (0) - register, not interested in readiness
   1 CURL_POLL_IN - register, interested in read readiness
   2 CURL_POLL_OUT - register, interested in write readiness
   3 CURL_POLL_IN | CURL_POLL_OUT - register, interested in both
   4 CURL_POLL_REMOVE - deregister

"handle" is used to uniquely identify the opaque structure associated
with a specific fd, if any (there won't be with select() although a
non-NULL pointer is still needed), so that the fdset doesn't have to
maintain a table mapping fd numbers to internal structures.

The "handle" argument is NULL when registering interest in a new fd,
or it is the value previously returned when modifying interest in an fd.
The "what" flags are no different in either case.

The return value is NULL on error (with errno set), or a non-NULL
handle associated with this fd.

A non-NULL return value should be supplied in all future callbacks,
until CURL_POLL_REMOVE is used: then a non-NULL return value indicates
success, but it is no longer a valid handle.

Example code called from curl_multi_perform():

Curl wants to wait for reading from a newly opened socket:

    void * handle = fdset_modify (fdset, NULL, socket_fd, CURL_POLL_IN,
                                  _curl_fd_callback, something);
    if (handle == NULL)
      report_error (...);
    curl_structures->handle = handle;

When curl decides to _change_ to a different combination of read/write
that it's interested in, it'd use the previously returned handle:

    handle = curl_structures->handle;
    if (fdset_modify (fdset, handle, socket_fd, CURL_POLL_IN | CURL_POLL_OUT)
                      _curl_fd_callback, something) == NULL)
      report_error (...);

or:

    handle = curl_structures->handle;
    if (fdset_modify (fdset, handle, socket_fd, CURL_POLL_OUT,
                      _curl_fd_callback, something) == NULL)
      report_error (...);

When curl is not waiting for anything from a socket, but the socket is
still kept open:

    handle = curl_structures->handle;
    if (fdset_modify (fdset, handle, socket_fd, CURL_POLL_NONE /* or 0 */,
                      _curl_fd_callback, something) == NULL)
      report_error (...);

When curl has finished with an fd, i.e. after closing a socket:

    handle = curl_structures->handle;
    if (fdset_modify (fdset, handle, socket_fd, CURL_POLL_REMOVE,
                      _curl_fd_callback, 0) == NULL)
      report_error (...);
    /* After this point, "handle" is no longer a valid value. */
    curl_structures->handle = NULL;

[ CURL_POLL_NONE and CURL_POLL_REMOVE are separated because some event
loop systems are more efficient if they can distinguish between full
deregistering and just temporary lack of interest. ]

The "callback" and "arg" arguments are used by fdset_wait(). For any
descriptors which are actually ready after calling select() or poll()
or something_else(), the app-supplied wait function will call the
curl-supplied callback in one of 3 ways:

    callback (arg, CURL_POLL_IN); - ready for read/accept
    callback (arg, CURL_POLL_OUT); - ready for write
    callback (arg, CURL_POLL_IN | CURL_POLL_OUT); - ready for both

This is how the wait function tells curl which fds are ready.

Naturally, curl will supply defaults for these fdset_* functions
which simple multi-socket apps will use.

A simple multi-socket app's main loop looks like:

   void * fdset = curl_fdset_init();
   if (fdset == NULL)
     report_error (...);
   curl_multi_init(args, fdset, curl_fdset_modify);
   while(still_things_to_do) {
     curl_multi_perform(args); /* do stuff and update fds */
     curl_fdset_wait(); /* wait on 10000 connections */
   }
   curl_fdset_destroy (fdset);

As you can see, this is nice and simple.

An app which uses a different event loop, e.g. Gnome's, will use
alternatives to those curl_fdset_* functions.

-- Jamie

(ps. For squeezing the most performance, you also want to distinguish
between waiting for read/write due to EAGAIN from recv/send, and
waiting due to short-recv/send.. This is due to some I/O event
primitives being edge driven (Linux RTsig) and others being level
driven (select...). But I don't want to confuse anyone further, and
it only saves system calls, it doesn't change the scaling behaviour).
Received on 2005-02-02