cURL / Mailing Lists / curl-library / Single Mail

curl-library

Re: Link between curl_easy handle and connection

From: Pierre Brico <pierre.brico_at_gmail.com>
Date: Mon, 7 Mar 2016 19:01:47 +0100

Okay, I see what you mean. You use the curl_multi API to wrap an easy
handle to avoid sharing connections between easy handles as written in curl
documentation:

"When you've created a multi handle and are using the multi interface, the
connection pool is instead kept in the multi handle so closing and creating
new easy handles to do transfers will not affect them. Instead all added
easy handles can take advantage of the single shared pool."

Unfortunately, I already use the multi interface to execute several
transfers in parallel. As I would like my tool to be as efficient as
possible, I use a pool of threads which manage the transfers. Each thread
is responsible of a certain number of easy handles and executes the
transfers using the curl_multi API in conjunction with the libevent library.

If I follow your suggestion, I will have one multi handle per transfer and
so one thread to manage it (waste of resources). This will prevent me to
execute thousands transfers in parallel (currently one thread is able to
process 100 connections).

Pierre

On Mon, Mar 7, 2016 at 6:14 PM, Isaac Boukris <iboukris_at_gmail.com> wrote:

> On Mon, Mar 7, 2016 at 3:30 PM, Pierre Brico <pierre.brico_at_gmail.com>
> wrote:
> >
> > Hi Isaac,
> >
> > To my knowledge, the multi interface allows just you to manipulate
> several
> > easy handles at a time (asynchronously) but doesn't give access to the
> > underlying connection to the server. Am I wrong ?
>
>
> What I meant is to wrap each easy handle with its own multi handle
> into a struct, then initialize the multi handle and set the
> socket/timeout callbacks, then init the easy handle and set the
> options and the read/write callbacks, then finally launch the transfer
> by calling curl_multi_add_handle (which will trigger an initial call
> to the timeout callback).
>
> I think it is somewhat similar to how the easy interface works, only
> asynchronous and non-blocking.
> -------------------------------------------------------------------
> List admin: https://cool.haxx.se/list/listinfo/curl-library
> Etiquette: https://curl.haxx.se/mail/etiquette.html

-------------------------------------------------------------------
List admin: https://cool.haxx.se/list/listinfo/curl-library
Etiquette: https://curl.haxx.se/mail/etiquette.html
Received on 2016-03-07