curl / Mailing Lists / curl-library / Single Mail
Buy commercial curl support from WolfSSL. We help you work out your issues, debug your libcurl applications, use the API, port to new platforms, add new features and more. With a team lead by the curl founder himself.

Re: LRU connection pooling can result in uneven load

From: Jeffrey Tolar via curl-library <curl-library_at_lists.haxx.se>
Date: Fri, 17 Sep 2021 08:17:03 -0700

On Sep 17 2021 12:14, Daniel Stenberg wrote:
>Thanks for your detailed description. I had to read it several times
>to fully understand your situation. I believe I fully understand it
>now!

Thanks for taking the time to do so!

>Back in the dark ages we actually had a CLOSEPOLICY option which would
>control with method libcurl would use to pick an old connection to
>close when the connection pool ran full. We removed it early on
>because it was never used and it was hard for users to understand the
>benefit of.

Ah, interesting! I can see how that would be more confusing for the
majority of use cases - one of the nice things about curl is that, for
the most part, it Just Works :).

>To me, a CURLOPT_MAXLIFETIME_CONN option seems like a better fit and
>it could go very well together with the existing CURLOPT_MAXAGE_CONN
>option (that sets the maximum idle time to allow a connection in the
>pool to reach). Connections that reach that age **while in the pool**
>will get pruned. That should prevent one or a few connections to get
>stuck on a fixed server for a very long time.

Great! I can try to work on a PR, probably either today or early next
week.

I haven't tried it yet, but I think the simplest implementation (at
least for me) would be to mirror CURLOPT_MAXAGE_CONN, and add an
additional check in conn_maxage [*]. That won't close connections
immediately when they expire, but I think it will be good enough for
most cases.

[*] https://github.com/curl/curl/blob/b0eda8dc6eb7be3d3e0762be5fb2a60989c2f77b/lib/url.c#L968-L981

-- 
Jeffrey Tolar
-- 
Unsubscribe: https://lists.haxx.se/listinfo/curl-library
Etiquette:   https://curl.haxx.se/mail/etiquette.html
Received on 2021-09-17