cURL / Mailing Lists / curl-library / Single Mail

curl-library

Re: Multithreaded use of libcurl

From: doa379 <doa379_at_gmail.com>
Date: Tue, 22 Sep 2015 15:21:46 +0100

>>>
>>>> On Sep 17, 2015, at 1:06 PM, doa379 <doa379_at_gmail.com> wrote:
>>>>
>>>> I think my issue has resolved itself by putting in a delay to some
>>>> of the operations in the threads.
>>>
>>> If you use delays to "fix" a bug in a threaded program, you are doing
>>> it wrong, and the bug will almost certainly resurface later to bite you.
>>>
>>> -Aaron
>>> -------------------------------------------------------------------
>>> List admin: http://cool.haxx.se/list/listinfo/curl-library
>>> Etiquette: http://curl.haxx.se/mail/etiquette.html
>>>
>>
>>
>> On 18/09/15 05:53, Ray Satiro via curl-library wrote:
>> >
>> > That doesn't sound right. I think your solution is possibly covering up
>> > a problem with the design of your program.
>> > -------------------------------------------------------------------
>> > List admin: http://cool.haxx.se/list/listinfo/curl-library
>> > Etiquette: http://curl.haxx.se/mail/etiquette.html
>>
>>
>> The program follows this example:
>>
>> http://softpixel.com/~cwright/programming/threads/threads.c.php
>
> The example has two threads and is using usleep like a yield to allow
> for a task switch but I don't think he adequately explains that. The
> order in which the output appears in your console is unpredictable,
> however it will usually alternate.
>
> "The output will be (mostly) alternating lines as the main() and
> threadFunc() threads execute and pause. Without the usleep()'s they'll
> not switch because we aren't doing anything that takes long enough to
> consume our whole time slice."
>
> He is trying to show how a scheduler will run threads by multiplexing
> them in time slices on a core. Scheduler says do some of this thread
> then time's up and scheduler says do some of that thread and so on. But
> the whole time slice isn't necessarily consumed and also both threads
> may be running at the same time in parallel truly concurrent if they're
> running on different cores. They may alternate because they're both
> calling printf and there is lock contention to obtain ownership of
> stdout. In other words one thread has to wait to write if the other is
> writing to the same. Also, usleep guarantees at a minimum suspension of
> n microseconds if no signal, not maximum so it could be any amount of
> microseconds >= n before a thread resumes.
>
> I think we are treading outside the scope of this list now but how have
> you followed the example in your program? My take so far is you have one
> thread that is performing the transfer using libcurl and receiving a
> continuous stream of records, and those records are then processed by
> other threads concurrently. I don't understand why you would fork the
> libcurl thread; also there seem to be some problems with forking and
> multi-threading. I am mostly in Windows so someone experienced in
> forking multi-threaded apps would be better to explain why you should or
> shouldn't do that.
>

Excuse me for the inappropriate use of the term "fork." The thread isn't
forked. Forking and Threading are distinct definitions.

Printing to stdout is massively inefficient. Every time that an
indefinite loop runs stdout the program is waiting to output to the
screen. The usleep() function just slows things down. But correct, it's
just an artefact introduced into the program to observe its behaviour.

I used the usleep() function initially to debug the program but have
since had no reason to keep it and have removed it.
-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-library
Etiquette: http://curl.haxx.se/mail/etiquette.html
Received on 2015-09-22