On Wed, 10 Aug 2011, Alex Loukissas wrote:

> I am looking to refactor this code now, so that I get a speedup by
> parallelism (the order that these requests are performed doesn't need to be
> FIFO or anything, as long as they all succeed), and connection reuse (i.e.
> filling up the pipe).

In many situations you won't gain any performance by doing parallell uploads
to a single server, and you'll get a simpler implementation by doing serial
uploads so I'd recommend that. At least for a first shot.

Just make sure you re-use the same CURL handle _without_ doing cleanup/init
between each individual transfer.

> One of the things I've looked at is going into the curl_multi_* interface,
> have a set number of handles and reuse them until all requests have been
> served. My question is the following: is there a benefit of doing this
> (which from what I understand is single-threaded and serialized --see:
> http://curl.haxx.se/mail/lib-2011-06/0065.html) over having a thread pool of
> curl_easy_handles that each has a connection to the server?

For a low degree of parallellism, you probably will not see any speed
differences between the multi interface and doing threaded easy interface
transfers. For a large number of simultaneous transfers (where large is at
least above 100), you should see better performance by going with the
multi_socket API. Mostly it is a matter of how you want to design your
software.

-- 
  / daniel.haxx.se
-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-library
Etiquette:  http://curl.haxx.se/mail/etiquette.html