curl / Mailing Lists / curl-library / Single Mail

curl-library

Re: Multithreading with libcurl

From: Ray Satiro via curl-library <curl-library_at_cool.haxx.se>
Date: Wed, 14 Dec 2016 22:56:00 -0500

On 12/14/2016 6:23 PM, Miloš Ljumović wrote:
> I wrote you before about my usage of multi stack among separate threads.
> It works fine, although I am keep getting output like "forcibly told
> to drain data" when using curl_multi_info_read.
> If that's fine, good - but I suspect that, something might be wrong
> and at some point app will crash.

Do you mean a crash might happen or it is happening? Forcible drain I
don't know a lot about. I think it happens when there is pending http2
decoded data available but at the same time the socket isn't readable
for some reason? Daniel will have a better idea about this.

> Recently I had to introduce reading the http response using
> write_callback which complicates my application even further, cause
> entire logic with multi interface is asynchronous and after I added
> the write_callback it required me introducing more structures and more
> synchronization mechanisms (to properly determine for which of
> concurrent requests write_callback was invoked).

The write callback is what is supposed to receive the body so I don't
understand the problem with that. You can use CURLOPT_WRITEDATA [1] to
pass a user pointer to the callback.

>
> I wanted to re-code it properly, if possible using the easy interface
> cause I assume that unlike when using multi interface, write_callback
> will be invoked (if http response exists) after calling
> curl_easy_perfom but prior to function returns. In other words entire
> operation will be synchronous. If so, few questions:
>
> 1. In https://curl.haxx.se/libcurl/c/threaded-ssl.html as well as on
> https://www.openssl.org/docs/crypto/threads.html#DESCRIPTION it says I
> should set up the openssl lock mechanisms. This bothers me:
>
> for(i=0; i<CRYPTO_num_locks(); i++) {
> pthread_mutex_init(&(lockarray[i]), NULL);
> }
>
> What I do not understand is why openssl requires more than one mutex?
> Isn't it one mutex (or CS) enough for synchronizing the threads, that
> will access shared resource? Why more than one? Are there more shared
> resources? Is it related to some openssl internal implementation?
>

OpenSSL document for 1.0.2 says that "locking_function() must be able to
handle up to CRYPTO_num_locks() different mutex locks" [2]. If you need
more information and nobody else here knows then ask OpenSSL. If you are
using OpenSSL 1.1.0 built for threading then you don't need those callbacks.

> 2. Important for my application was that connections are re-used,
> otherwise Apple Push Notification service would consider my requests
> (if there are a lot of them) as DoS attacks. This is the reason I
> decided to use "multi interface". Question is if I use easy interface,
> let's say prepare everything on easy handle, call the
> curl_easy_perform then curl_easy_reset, and then set up the handle
> again - in following call to curl_easy_perform with the same domain
> (https://apple.com/...) but different page will the same connection be
> used? If yes, easy interface will do fine for me.

Yes. In most circumstances libcurl will keep the connection open and
attempt to reuse it, read the overview [3].

[1]: https://curl.haxx.se/libcurl/c/CURLOPT_WRITEDATA.html
[2]: https://www.openssl.org/docs/man1.0.2/crypto/threads.html
[3]: https://ec.haxx.se/libcurl-connectionreuse.html

-------------------------------------------------------------------
List admin: https://cool.haxx.se/list/listinfo/curl-library
Etiquette: https://curl.haxx.se/mail/etiquette.html
Received on 2016-12-15