curl / Mailing Lists / curl-library / Single Mail

curl-library

Re: curl handles and multi thread

From: surya chandrika via curl-library <curl-library_at_cool.haxx.se>
Date: Tue, 21 Nov 2017 13:39:48 +0530

Hi All,

Actually i have multiple message queues say for eg 50 queues(shared bw
threads) with messages that needs to be delivered without affecting the
sequence of messages in a particular queue.

Multiple threads are supposed to serve this purpose.
Each thread keeps polling this message queue ,which ever is open,thread
takes a message and deliver them one by one.

in current implementation its a sync request so if the destination host is
down, this thread has a waiting period, so in order to make them faster,
we planning to switch to async where thread_a (from a pool of threads)
perform multi_perform when a message queue is available and puts them to a
pending messages queue. which is then processed by other thread_b(from a
pool of threads) once ready.
This thread_a will now be open to poll the other message queues. so handles
prcoessed by multiple threads would be different , so i assume threads
should not be blocking except for the select()/timeout in thread_b.

Please share your thoughts for the same.

Thanks
Surya

On Sat, Nov 11, 2017 at 3:52 AM, Daniel Stenberg <daniel_at_haxx.se> wrote:

> On Wed, 8 Nov 2017, surya chandrika via curl-library wrote:
>
> You must never share the same handle in multiple threads. You can pass the
>> handles around among threads, but you must never use a single handle from
>> more than one thread at any given time.
>>
>> Usecase: (cpp)
>>
>
> 1. Get a multi handle and post an async req
>> 2. If still_running = 1 push them into a global map
>>
>
> Them?
>
> 3. another thread which actively parse this map to select() the handles.
>> (one thread dedicated for selection for pending request)
>>
>
> Why another thread? The thread in (2) can't do anything anyway as long as
> (3) uses the same handle.
>
> 4. If an activity is detected another thread from threadpool perform multi
>> perform.
>>
>
> Yet another? It's still the same multi handle,so it can't do anything with
> the handle unless the other threads using the handle all hold off.
>
> So using a proper locking mechanism i understand that i should be able to
>> perform the above without any exception.
>>
>
> Sure, but the question is then left why you feel you need to use so many
> threads...
>
> --
>
> / daniel.haxx.se
>

-------------------------------------------------------------------
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library
Etiquette: https://curl.haxx.se/mail/etiquette.html
Received on 2017-11-21