Buy commercial curl support from WolfSSL. We help you work
out your issues, debug your libcurl applications, use the API, port to new
platforms, add new features and more. With a team lead by the curl founder
himself.
Re: Epoll performance issues.
- Contemporary messages sorted: [ by date ] [ by thread ] [ by subject ] [ by author ] [ by messages with attachments ]
From: Daniel Stenberg via curl-library <curl-library_at_cool.haxx.se>
Date: Fri, 27 Nov 2020 08:22:49 +0100 (CET)
On Fri, 27 Nov 2020, James Read wrote:
> Has anybody ever actually succeeded in making a high performance application
> with epoll/libcurl as the back end.
Yes. Although most people I know of use an event library in between and not
epoll directly, but they would still eventually use epoll on Linux machines.
> At this point I really only see two possibilities:
If any of those actually are true, then surely you should be able to create a
smaller example showing how to repro the issue?
> As the code I have presented only applies minimal changes to the example
> code at https://curl.se/libcurl/c/ephiperfifo.html I guess we have to also
> consider the possibility that this code is buggy as well.
Sure, that can very well be true. It's just an example and we've polished
details in it before. Maybe there's something else in there not done
optimally?
I think the key to figuring that out is to 1) understand the libcurl API *and*
2) understand the epoll API and use. I personally am very weak on that second
point.
> this is what leads me to believe there is a problem with the way libcurl
> handles the creation of many new multiple connections when used with epoll.
libcurl doesn't use epoll itself and isn't even aware of epoll and it will
create and use multiple connection exactly the same completely independent of
the underlying event system that it isn't aware of.
> Running the two codes to see the drop of performance would only take a few
> minutes of anybody's time. A simple diff between these proofs and the
> example code at https://curl.se/libcurl/c/ephiperfifo.html would verify
> that the changes are minimal.
If you think you can reproduce an issue and you can provide the source code
for it in a smallish application source code, then please do so. It would
allow us all to run, test and debug the thing from our ends.
Have you said what libcurl version and SSL backend you're using? I presume
most of your URLs are HTTPS?
Date: Fri, 27 Nov 2020 08:22:49 +0100 (CET)
On Fri, 27 Nov 2020, James Read wrote:
> Has anybody ever actually succeeded in making a high performance application
> with epoll/libcurl as the back end.
Yes. Although most people I know of use an event library in between and not
epoll directly, but they would still eventually use epoll on Linux machines.
> At this point I really only see two possibilities:
If any of those actually are true, then surely you should be able to create a
smaller example showing how to repro the issue?
> As the code I have presented only applies minimal changes to the example
> code at https://curl.se/libcurl/c/ephiperfifo.html I guess we have to also
> consider the possibility that this code is buggy as well.
Sure, that can very well be true. It's just an example and we've polished
details in it before. Maybe there's something else in there not done
optimally?
I think the key to figuring that out is to 1) understand the libcurl API *and*
2) understand the epoll API and use. I personally am very weak on that second
point.
> this is what leads me to believe there is a problem with the way libcurl
> handles the creation of many new multiple connections when used with epoll.
libcurl doesn't use epoll itself and isn't even aware of epoll and it will
create and use multiple connection exactly the same completely independent of
the underlying event system that it isn't aware of.
> Running the two codes to see the drop of performance would only take a few
> minutes of anybody's time. A simple diff between these proofs and the
> example code at https://curl.se/libcurl/c/ephiperfifo.html would verify
> that the changes are minimal.
If you think you can reproduce an issue and you can provide the source code
for it in a smallish application source code, then please do so. It would
allow us all to run, test and debug the thing from our ends.
Have you said what libcurl version and SSL backend you're using? I presume
most of your URLs are HTTPS?
-- / daniel.haxx.se | Commercial curl support up to 24x7 is available! | Private help, bug fixes, support, ports, new features | https://www.wolfssl.com/contact/ ------------------------------------------------------------------- Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library Etiquette: https://curl.se/mail/etiquette.htmlReceived on 2020-11-27