Buy commercial curl support from WolfSSL. We help you work
out your issues, debug your libcurl applications, use the API, port to new
platforms, add new features and more. With a team lead by the curl founder
himself.
Re: Re: Epoll performance issues.
- Contemporary messages sorted: [ by date ] [ by thread ] [ by subject ] [ by author ] [ by messages with attachments ]
From: Patrick Schlangen via curl-library <curl-library_at_cool.haxx.se>
Date: Fri, 27 Nov 2020 08:56:17 +0100
On Fri, 27 Nov 2020, James Read wrote:
> Has anybody ever actually succeeded in making a high performance application
> with epoll/libcurl as the back end.
Yes. Some things I've learned (for my usecase):
* If using SSL hosts, set CURLOPT_CAINFO to a small list of accepted CAs or, if not interested in validation, to NULL. In my case, the default list of trusted CAs which came with my OS (Ubuntu) was loaded all the time for every connection, and it was huge and that caused a significant (!) performance drop for the whole application.
* Don't try to built your own c-ares integration or so (I think I've seen something like this when briefly looking at your code). Just build libcurl with c-ares support. Don't use any other resolver for high performance async applications.
* Disable IPv6 if not needed. (CURLOPT_IPRESOLVE = CURL_IPRESOLVE_V4)
* Disable following of redirections if not needed.
* Use libev. All other event libraries didn't work reliably for me.
* Build libcurl, c-ares, openssl etc from source using latest stable versions. Ensure you really build/link against the versions you built yourself. Stuff coming with your OS often is outdated or not configured for your usecase (in libcurl case e.g. using a different ssl lib and/or resolver).
* Profile your application. Depending on the bottleneck, on multi core systems, it might make sense to spawn multiple threads with their own curl multi instances.
Maybe this helps.
Patrick
-------------------------------------------------------------------
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library
Etiquette: https://curl.se/mail/etiquette.html
Received on 2020-11-27
Date: Fri, 27 Nov 2020 08:56:17 +0100
On Fri, 27 Nov 2020, James Read wrote:
> Has anybody ever actually succeeded in making a high performance application
> with epoll/libcurl as the back end.
Yes. Some things I've learned (for my usecase):
* If using SSL hosts, set CURLOPT_CAINFO to a small list of accepted CAs or, if not interested in validation, to NULL. In my case, the default list of trusted CAs which came with my OS (Ubuntu) was loaded all the time for every connection, and it was huge and that caused a significant (!) performance drop for the whole application.
* Don't try to built your own c-ares integration or so (I think I've seen something like this when briefly looking at your code). Just build libcurl with c-ares support. Don't use any other resolver for high performance async applications.
* Disable IPv6 if not needed. (CURLOPT_IPRESOLVE = CURL_IPRESOLVE_V4)
* Disable following of redirections if not needed.
* Use libev. All other event libraries didn't work reliably for me.
* Build libcurl, c-ares, openssl etc from source using latest stable versions. Ensure you really build/link against the versions you built yourself. Stuff coming with your OS often is outdated or not configured for your usecase (in libcurl case e.g. using a different ssl lib and/or resolver).
* Profile your application. Depending on the bottleneck, on multi core systems, it might make sense to spawn multiple threads with their own curl multi instances.
Maybe this helps.
Patrick
-------------------------------------------------------------------
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library
Etiquette: https://curl.se/mail/etiquette.html
Received on 2020-11-27