Disclaimer: these ideas are not yet fully considered or even likely to be how
this will end up like. Feel free to object, fill in, complain or suggest
otherwise. I'm just posting these ideas at an early stage for you to have a
chance to participate. Some of this comes from earlier posts, from me and
others.
   ---------- snip ----------------
GOAL

  To introduce a way for libcurl to do multiple file transfers using the same
  connection.

HOW

  curl_easy_perform() would
  - not disconnect from the server when one file has been transfered, it
    leaves the connection "open" for further use
  - re-use an earlier opened connection if one exist to the server we are
    about to transfer a document to/from, the re-usage would deal with the
    fact that the connection might have timed out or similar
  - open more connections and keeep the old one(s) open if the new document
    needs a new connection
  - have a maximum amount of simultaneously opened connections and close the
    oldest when that amount is reached

  1. An option to determine how many connections can be opened simultaneously.
     If zero, connections will be closed after each transfer (as today).

     CURLOPT_MAXCONNECTIONS - default is ... like 5?

     Note that the current interface *requires* the user to init/ perform/
     cleanup for every transfer, so there's no way old programs can be made to
     automatically transform to use persistant connections just by upgrading
     the library. But by using sensible defaults, the only change needed would
     be to move the init() and cleanup() calls to outside the loop and do them
     only once.

  2. An option to determine the strategy used to close persistant connections.
     Strategies can be to close
       o the oldest (from its creation date)
       o the least recently used
       o the one with least traffic
       o round-robin, random, etc.

     CURLOPT_CLOSINGPOLICY - default is CURLCLOSEPOLICY_OLDEST
        CURLCLOSEPOLICY_OLDEST,
        CURLCLOSEPOLICY_LEAST_RECENTLY_USED
        CURLCLOSEPOLICY_LEAST_TRAFFIC,
        CURLCLOSEPOLICY_RANDOM

USAGE

         This could turn out to be working pseudo code:

 curl_easy_init()

 [ a number of ... ] curl_easy_setopt()

 do {

   curl_easy_setopt( URL )

   curl_easy_perform();

 } while( another URL to transfer );

 curl_easy_cleanup()

DESIGN ISSUES

  Jotted down thoughts and ideas on implementation details

  * How to find an already existing connection to use?
   - identical host name (we don't want to do any host lookups)
   - identical port number
   - identical protocol

    Protocol dependent match condtions:
   - (FTP) identical user+passwd, as it is set for the connection in a way
     that differs from HTTP.

    Note: a connection through a HTTP proxy is likely to have all of these
    things in common for almost any remote document, and thus will be much
    more likely to actually use the same connection multiple times.

  * Linked list/reallocated array for keeping connections?

  * Make sure that the Location: following engine uses the multiple connection
    policy just as all connections.

  * We should safely rely on the fact that the function sequence will *always*
    only be one function invoked at a single time using the same curl handle.
    This will be documented, any simultaneous use of the same handle will give
    unknown results.

-- 
  Daniel Stenberg -- curl project maintainer -- http://curl.haxx.se/
_______________________________________________
Curl-library mailing list
Curl-library_at_lists.sourceforge.net
http://lists.sourceforge.net/lists/listinfo/curl-library