Buy commercial curl support from WolfSSL. We help you work
out your issues, debug your libcurl applications, use the API, port to new
platforms, add new features and more. With a team lead by the curl founder
himself.
Re: Re: Epoll performance issues.
- Contemporary messages sorted: [ by date ] [ by thread ] [ by subject ] [ by author ] [ by messages with attachments ]
From: James Read via curl-library <curl-library_at_cool.haxx.se>
Date: Sun, 29 Nov 2020 14:16:47 +0000
Hi,
On Fri, Nov 27, 2020 at 7:56 AM Patrick Schlangen <patrick_at_schlangen.me>
wrote:
> On Fri, 27 Nov 2020, James Read wrote:
> > Has anybody ever actually succeeded in making a high performance
> application
> > with epoll/libcurl as the back end.
>
> Yes. Some things I've learned (for my usecase):
>
What's your usecase? What kind of throughput were you able to achieve.
>
> * If using SSL hosts, set CURLOPT_CAINFO to a small list of accepted CAs
> or, if not interested in validation, to NULL. In my case, the default list
> of trusted CAs which came with my OS (Ubuntu) was loaded all the time for
> every connection, and it was huge and that caused a significant (!)
> performance drop for the whole application.
>
> * Don't try to built your own c-ares integration or so (I think I've seen
> something like this when briefly looking at your code). Just build libcurl
> with c-ares support. Don't use any other resolver for high performance
> async applications.
>
> * Disable IPv6 if not needed. (CURLOPT_IPRESOLVE = CURL_IPRESOLVE_V4)
>
> * Disable following of redirections if not needed.
>
> * Use libev. All other event libraries didn't work reliably for me.
>
> * Build libcurl, c-ares, openssl etc from source using latest stable
> versions. Ensure you really build/link against the versions you built
> yourself. Stuff coming with your OS often is outdated or not configured for
> your usecase (in libcurl case e.g. using a different ssl lib and/or
> resolver).
>
> * Profile your application. Depending on the bottleneck, on multi core
> systems, it might make sense to spawn multiple threads with their own curl
> multi instances.
>
>
Thanks for the tips. Have implemented some of them already with some
limited success. Will keep working through the list.
James Read
> Maybe this helps.
>
> Patrick
>
>
>
-------------------------------------------------------------------
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library
Etiquette: https://curl.se/mail/etiquette.html
Received on 2020-11-29
Date: Sun, 29 Nov 2020 14:16:47 +0000
Hi,
On Fri, Nov 27, 2020 at 7:56 AM Patrick Schlangen <patrick_at_schlangen.me>
wrote:
> On Fri, 27 Nov 2020, James Read wrote:
> > Has anybody ever actually succeeded in making a high performance
> application
> > with epoll/libcurl as the back end.
>
> Yes. Some things I've learned (for my usecase):
>
What's your usecase? What kind of throughput were you able to achieve.
>
> * If using SSL hosts, set CURLOPT_CAINFO to a small list of accepted CAs
> or, if not interested in validation, to NULL. In my case, the default list
> of trusted CAs which came with my OS (Ubuntu) was loaded all the time for
> every connection, and it was huge and that caused a significant (!)
> performance drop for the whole application.
>
> * Don't try to built your own c-ares integration or so (I think I've seen
> something like this when briefly looking at your code). Just build libcurl
> with c-ares support. Don't use any other resolver for high performance
> async applications.
>
> * Disable IPv6 if not needed. (CURLOPT_IPRESOLVE = CURL_IPRESOLVE_V4)
>
> * Disable following of redirections if not needed.
>
> * Use libev. All other event libraries didn't work reliably for me.
>
> * Build libcurl, c-ares, openssl etc from source using latest stable
> versions. Ensure you really build/link against the versions you built
> yourself. Stuff coming with your OS often is outdated or not configured for
> your usecase (in libcurl case e.g. using a different ssl lib and/or
> resolver).
>
> * Profile your application. Depending on the bottleneck, on multi core
> systems, it might make sense to spawn multiple threads with their own curl
> multi instances.
>
>
Thanks for the tips. Have implemented some of them already with some
limited success. Will keep working through the list.
James Read
> Maybe this helps.
>
> Patrick
>
>
>
-------------------------------------------------------------------
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library
Etiquette: https://curl.se/mail/etiquette.html
Received on 2020-11-29