Buy commercial curl support from WolfSSL. We help you work
out your issues, debug your libcurl applications, use the API, port to new
platforms, add new features and more. With a team lead by the curl founder
himself.
Re: LRU connection pooling can result in uneven load
- Contemporary messages sorted: [ by date ] [ by thread ] [ by subject ] [ by author ] [ by messages with attachments ]
From: Daniel Stenberg via curl-library <curl-library_at_lists.haxx.se>
Date: Fri, 17 Sep 2021 12:14:14 +0200 (CEST)
On Thu, 16 Sep 2021, Jeffrey Tolar via curl-library wrote:
Thanks for your detailed description. I had to read it several times to fully
understand your situation. I believe I fully understand it now!
libcurl's preference in picking the most recently used connection for reuse is
of course because in general, that's the connection with the highest
likeliness of not causing any problem when re-using.
Back in the dark ages we actually had a CLOSEPOLICY option which would control
with method libcurl would use to pick an old connection to close when the
connection pool ran full. We removed it early on because it was never used and
it was hard for users to understand the benefit of.
> Any thoughts on a new option to tweak the pooling behavior? Perhaps a
> CURLOPT_MAXLIFETIME_CONN setting, or a parameter for TODO 1.23 to be able to
> selectively flush connections from the pool?
In a case like you describe here, selectively flushing connections from the
pool seems like a rather difficult task for the application to do with any
accuracy.
To me, a CURLOPT_MAXLIFETIME_CONN option seems like a better fit and it could
go very well together with the existing CURLOPT_MAXAGE_CONN option (that sets
the maximum idle time to allow a connection in the pool to reach). Connections
that reach that age **while in the pool** will get pruned. That should prevent
one or a few connections to get stuck on a fixed server for a very long time.
Date: Fri, 17 Sep 2021 12:14:14 +0200 (CEST)
On Thu, 16 Sep 2021, Jeffrey Tolar via curl-library wrote:
Thanks for your detailed description. I had to read it several times to fully
understand your situation. I believe I fully understand it now!
libcurl's preference in picking the most recently used connection for reuse is
of course because in general, that's the connection with the highest
likeliness of not causing any problem when re-using.
Back in the dark ages we actually had a CLOSEPOLICY option which would control
with method libcurl would use to pick an old connection to close when the
connection pool ran full. We removed it early on because it was never used and
it was hard for users to understand the benefit of.
> Any thoughts on a new option to tweak the pooling behavior? Perhaps a
> CURLOPT_MAXLIFETIME_CONN setting, or a parameter for TODO 1.23 to be able to
> selectively flush connections from the pool?
In a case like you describe here, selectively flushing connections from the
pool seems like a rather difficult task for the application to do with any
accuracy.
To me, a CURLOPT_MAXLIFETIME_CONN option seems like a better fit and it could
go very well together with the existing CURLOPT_MAXAGE_CONN option (that sets
the maximum idle time to allow a connection in the pool to reach). Connections
that reach that age **while in the pool** will get pruned. That should prevent
one or a few connections to get stuck on a fixed server for a very long time.
-- / daniel.haxx.se | Commercial curl support up to 24x7 is available! | Private help, bug fixes, support, ports, new features | https://curl.se/support.html -- Unsubscribe: https://lists.haxx.se/listinfo/curl-library Etiquette: https://curl.haxx.se/mail/etiquette.htmlReceived on 2021-09-17