cURL / Mailing Lists / curl-library / Single Mail

curl-library

RE: single thread with multi or multiple threads with easy+share+openssl locking?

From: Yehezkel Horowitz <horowity_at_checkpoint.com>
Date: Thu, 27 Feb 2014 12:16:43 +0000

>We would like to wrap libcurl into some queuing/threading for easy usage in various applications and we were wonderting what precisely the (dis)advantage are to using either one or the other technique...

>I.e. we have a lot of URL's that will need querying and are wondering if we should work down our queue of urls by using either one thread for each URL, with a possibly re-cycled easy running in that thread, or one single thread to do all URL's, but rather use a multi and feed it new easy objects at the right time.

>Are there any additional pitfalls we need to be aware of except non-shared data (which could be fixed with the share interface as far as I understand it) and the openssl locking...

I assume there might be other things, but the first come to my mind is the reuse of the connections.

If some of your URLs are to the same host:port, you can benefit the re-usage of TCP connections which is seamlessly managed by curl_multi_ API (of course you can disable the re-use if needed).

With your multi-threaded approach, you'll need to assign URLs to the same host:port to the same thread in order to re-use connections ( = manage the re-use by yourself).

Regards,

Yehezkel Horowitz

-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-library
Etiquette: http://curl.haxx.se/mail/etiquette.html
Received on 2014-02-27