curl / Mailing Lists / curl-library / Single Mail

curl-library

Re: Asynchronous ordered sends

From: Ray Satiro via curl-library <curl-library_at_cool.haxx.se>
Date: Thu, 23 Mar 2017 15:52:35 -0400

On 3/23/2017 8:09 AM, Richard Gray wrote:
> Daniel Stenberg wrote:
>> On Tue, 21 Mar 2017, Emir Kamber wrote:
>>
>>> I have successfully implemented async sends via 'multi-functionality',
>>> but I have a problem where messages are sent in no particular order.
>>> So,
>>> when I send 3 messages, I get this on the wire: MSG3, MSG1, MSG2.
>>>
>>> I need to send messages in the order that I add them via
>>> 'curl_multi_add_handle', so that I get MSG1, MSG2, MSG3 on the wire. In
>>> essence I just need to send an X number of messages in sequence as
>>> quickly as possible, and then will worry about reading the response
>>> from
>>> the server.
>>
>> The multi API handles many transfers in parallel, but it will do them
>> all
>> independent and as fast as possible. If you add three easy handles,
>> libcurl
>> will try to perform those three transfers as quickly as it can.
>>
>> If you need the transfers done serially, you need to wait until
>> transfer one
>> is done before you add the easy handle for transfer two and so on.
>>
>
> If one wants to have messages transmit in order, but with overlap, like
>
> MSG111111111111
> MSG22222222222222
> MSG333333333333333
>
> I wonder if a suitable callback from msg1 transfer could be used to
> fire off msg2 once msg1 is underway (first read callback?) A
> callback from msg2 could launch msg3 once msg2 is going...
>
> Is it safe to add a new easy handle to a multi handle in a read or
> write callback or would some other technique have to be used?
> Connection re-use could complicate things...

I'm not convinced. If you send messages asynchronously like that there's
no guarantee they'll be processed by the server in that order, so why do it.

Let's say the OP limits to one connection in the multi via
CURLMOPT_MAX_TOTAL_CONNECTIONS [1] and then libcurl does the transfers
in the order they were added [2][3], and the waiting handles will be
held in limbo between CONNECT_PEND and CONNECT states. I'm pretty sure
that is not currently documented therefore it's not guaranteed. Even so,
I suppose a number of things (timeout, transfer failed, etc) could cause
it to just go to the next handle.

In other words I don't think there's a "depends on" type logic to check
if multi with A B C D already added and A transfer successful then it's
OK to send B transfer (and repeat for C, D, etc) otherwise stop.

[1]: https://curl.haxx.se/libcurl/c/CURLMOPT_MAX_TOTAL_CONNECTIONS.html
[2]: https://github.com/curl/curl/blob/curl-7_53_1/lib/multi.c#L423-L436
[3]: https://github.com/curl/curl/blob/curl-7_53_1/lib/multi.c#L2158-L2171

-------------------------------------------------------------------
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library
Etiquette: https://curl.haxx.se/mail/etiquette.html
Received on 2017-03-23