Buy commercial curl support from WolfSSL. We help you work
out your issues, debug your libcurl applications, use the API, port to new
platforms, add new features and more. With a team lead by the curl founder
himself.
Re: LRU connection pooling can result in uneven load
- Contemporary messages sorted: [ by date ] [ by thread ] [ by subject ] [ by author ] [ by messages with attachments ]
From: Jeffrey Tolar via curl-library <curl-library_at_lists.haxx.se>
Date: Fri, 17 Sep 2021 08:17:03 -0700
On Sep 17 2021 12:14, Daniel Stenberg wrote:
>Thanks for your detailed description. I had to read it several times
>to fully understand your situation. I believe I fully understand it
>now!
Thanks for taking the time to do so!
>Back in the dark ages we actually had a CLOSEPOLICY option which would
>control with method libcurl would use to pick an old connection to
>close when the connection pool ran full. We removed it early on
>because it was never used and it was hard for users to understand the
>benefit of.
Ah, interesting! I can see how that would be more confusing for the
majority of use cases - one of the nice things about curl is that, for
the most part, it Just Works :).
>To me, a CURLOPT_MAXLIFETIME_CONN option seems like a better fit and
>it could go very well together with the existing CURLOPT_MAXAGE_CONN
>option (that sets the maximum idle time to allow a connection in the
>pool to reach). Connections that reach that age **while in the pool**
>will get pruned. That should prevent one or a few connections to get
>stuck on a fixed server for a very long time.
Great! I can try to work on a PR, probably either today or early next
week.
I haven't tried it yet, but I think the simplest implementation (at
least for me) would be to mirror CURLOPT_MAXAGE_CONN, and add an
additional check in conn_maxage [*]. That won't close connections
immediately when they expire, but I think it will be good enough for
most cases.
[*] https://github.com/curl/curl/blob/b0eda8dc6eb7be3d3e0762be5fb2a60989c2f77b/lib/url.c#L968-L981
Date: Fri, 17 Sep 2021 08:17:03 -0700
On Sep 17 2021 12:14, Daniel Stenberg wrote:
>Thanks for your detailed description. I had to read it several times
>to fully understand your situation. I believe I fully understand it
>now!
Thanks for taking the time to do so!
>Back in the dark ages we actually had a CLOSEPOLICY option which would
>control with method libcurl would use to pick an old connection to
>close when the connection pool ran full. We removed it early on
>because it was never used and it was hard for users to understand the
>benefit of.
Ah, interesting! I can see how that would be more confusing for the
majority of use cases - one of the nice things about curl is that, for
the most part, it Just Works :).
>To me, a CURLOPT_MAXLIFETIME_CONN option seems like a better fit and
>it could go very well together with the existing CURLOPT_MAXAGE_CONN
>option (that sets the maximum idle time to allow a connection in the
>pool to reach). Connections that reach that age **while in the pool**
>will get pruned. That should prevent one or a few connections to get
>stuck on a fixed server for a very long time.
Great! I can try to work on a PR, probably either today or early next
week.
I haven't tried it yet, but I think the simplest implementation (at
least for me) would be to mirror CURLOPT_MAXAGE_CONN, and add an
additional check in conn_maxage [*]. That won't close connections
immediately when they expire, but I think it will be good enough for
most cases.
[*] https://github.com/curl/curl/blob/b0eda8dc6eb7be3d3e0762be5fb2a60989c2f77b/lib/url.c#L968-L981
-- Jeffrey Tolar -- Unsubscribe: https://lists.haxx.se/listinfo/curl-library Etiquette: https://curl.haxx.se/mail/etiquette.htmlReceived on 2021-09-17