Hi Daniel:

I owe you an apology. This problem is not due to any
problem with the libcurl library and would occur even
if was easy to access file and socket descriptors. The
problem is due to bad logic on my part, and being too
zealous in avoiding timers.

Daniel Stenberg wrote:

> On Fri, 9 Jul 2010, Paul Romero wrote:
>
> >> If you use curl_multi_perform() you don't want/need libevent. If you want
> >> libevent-style processing, you don't use curl_multi_perform().
> >
> > OK, I believe you. What alternative routine do you suggest which does not
> > rely upon the libevent library and FIFOs ?
>
> I'd suggest you first use the regular libcurl multi interface and get an
> understanding for how it works before you try more advanced or special
> approaches.

The problem occurred due to the following stupid assumption
about socket and file descriptors. The assumption is that
they can be programmed to generate an event when there
is a transition from output being blocked to not being
blocked. This is not true for any Linux/Unix drivers
I know of, although the fcntl() documentation suggests
it is true. Of course life would be easier if the drivers
generated such events, but there is no choice but to
deal with reality. The pseudo-code below is a recipe
for dealing with the problem without excessive CPU
usage and minimal dependency on timers.

>
>
> > 1) Send a single file between my system and another via SFTP.
> > 2) Perform the transaction asynchronously so that my program
> > can do other things during the file transfer.
>
> That's what the multi interface already provides. You just need to understand
> that asynchronous means non-blocking, so it won't complete anything in the
> background or so. You will need to call libcurl over and over until the
> transfer is complete. An example showing a single transfer done non-blocking
> is here: http://curl.haxx.se/libcurl/c/multi-single.html

Note, I studied that example and understand it.

>
>
> > Ideally, it would be possible to invoke a routine to start the file
> > transfer, and a handler indicating the outcome would execute when it is
> > complete. However, that may not be a reasonable expectation. I would expect
> > the library has some way to inform a program when data can be sent and the
> > program needs some code to respond to the notification.
>
> If you want something to deal with the entire transfer and then tell you when
> its done, then you need to do that transfer in a separate thread and then I'd
> say it would make sense to just use the easy interface there.
>

***************** RECIPE ********************

The logic of the following pseudo-code works for a wide variety
of applications and is OS independent. It consists of the following
procedures and enables a task using the logic to give
up the CPU without danger of a deadlock when it has
nothing to do. It only uses one timer to deal with
blocked output. That timer is set when output is blocked
and the task has nothing to do. When the timer expires,
the task retries the blocked output operation. Input events
and insertion of messages in the task's input queues cause
the timer to be cancelled. The logic requires the OS environment
to generate an event when input is available, and when a message is
inserted in a task's input queue if it has one.

Wait - Waits for an event when there is nothing to do.
Input_Event - Invoked when an input event is detected.
Output_Blocked - Invoked when output is blocked.
Start_Work - Invoked when a task begins performing
a subtask whose completion may require multiple invocations
and thus rescheduling of the main task.
End_Work(Finished) - Invoked to indicate the status of a
subtask started with an invocation of Start_Work. The
Finished parameter indicates the subtask is complete.

The following variables are shared by the procedures:

input_events - The number of unserviced input events detected.
output_blocked - TRUE when blocked output is detected and FALSE
otherwise.
more_work - The number of incomplete non-I/O subtasks. This
is used for situations in which the main task must be rescheduled
to complete a subtask.
queued_events - If the task has a message input queue, this is
TRUE when there are messages in the queue and FALSE otherwise.

Wait: This procedure is invoked at the beginning of the main
loop of a task. Note that when the CPU is relinquished, any of the
following events cause the task to resume: timer expiration,
an input event, insertion of a message in the task's input
queue. The queued_events test should not be used if the
task does not have a message input queue.

   wait <- TRUE
   If input_events
      wait <- FALSE
   Else
   If queued_events
      wait <- FALSE
   else
   If more_work
      wait <- FALSE
   Else
   If Not output_blocked
      wait <- FALSE

   If wait
      If output_blocked
         Set output_blocked timer.
      Wait for a timer or event (i.e. Give up the CPU. )
      Cancel the output_blocked timer.

   output_blocked <- FALSE
   If input_events
      input_events <- input_events - 1

Input_Event: This procedure is invoked each time an input
event is detected.
   input_event <- input_event + 1

Output_Blocked: This procedure is called when a module completes
and output is blocked.
   output_blocked <- TRUE

Start_Work: This procedure is invoked each time a module
begins a non-I/O subtask which it may not be able to finish
within the current invocation of the main task.
   more_work <- more_work + 1

End_Work(Finished): This procedure is invoked by a module
when upon completion of a subtask for which Start_Work()
was invoked. If Finished is TRUE the subtask is finished.
Otherwise, the module must be invoked again to complete
the subtask.
   if(Finished)
      more_work <- more_work - 1;

Best Regards,

Paul R.

>
> --
>
> / daniel.haxx.se
> -------------------------------------------------------------------
> List admin: http://cool.haxx.se/list/listinfo/curl-library
> Etiquette: http://curl.haxx.se/mail/etiquette.html

--
Paul Romero
RCOM Communications Software
Phone/Fax: (510)339-2628
E-Mail: paulr_at_rcom-software.com

-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-library
Etiquette: http://curl.haxx.se/mail/etiquette.html
Received on 2010-07-10