In the example "evhiperfifo.c", a named pipe is created and monitored.
When new URLs are sent to the pipe, line by line, then for every URL a
cURL easy handle is created.

Now, in want to change this process a little bit:

* I want to have URLs read from a database. So, naturally, there would
  be no monitoring of a named pipe.

* I want to only have a certain number of easy handles. This is in order
  not to exceed the system's limits, and in order not to make individual
  transfers too slow.

My idea:

  Every time after a call to "curl_multi_socket_action()", a MySQL
  database is checked for new URLs. Then, for each URL, a new easy
  handle is created, until a certain maximum number of handles has been
  reached.

  Background info: There is a separate process that times out URLs in
  the database, in order to prevent the queue from flowing over.

Questions:

* Is there any problem if program flow is paused right after calling
  "curl_multi_socket_action"?

* During connection to database (possibly slow), downloads will continue
  "in the background" under UNIX, right?
-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-library
Etiquette: http://curl.haxx.se/mail/etiquette.html
Received on 2012-06-10