Re: Guidance on monitoring connection open/close with CURLM API

From: Stefan Eissing via curl-library <curl-library_at_lists.haxx.se>
Date: Fri, 31 May 2024 08:54:57 +0200

> Am 31.05.2024 um 06:30 schrieb Cao Duc Quan via curl-library <curl-library_at_lists.haxx.se>:
>
> Hi,
>
> Here is the main flow in my application with CURLM
>
> struct Request {
> CURL *easy;
> // other data
> }
>
> int on_close_socket(void* clientp, curl_socket_t sock) {
> struct Request *request = (struct Request*)clientp;
> // log some metrics for request
> close(sock);
> return 0;
> }
>
> struct Request* alloc_request(const char* url) {
> struct Request* request = malloc(sizeof(struct Request));
>
> request->easy = curl_easy_init();
> curl_easy_setopt(curl, CURLOPT_CLOSESOCKETFUNCTION, on_close_socket);
> curl_easy_setopt(curl, CURLOPT_CLOSESOCKETDATA, request);
> // other init
> }
>
> void close_request(struct Request* request) {
> curl_easy_cleanup(request->easy);
> free(request);
> }
>
> struct Request* lkup_request(CURL *easy) {
> // lookup request from easy in active queue
> }
>
> void cleanup_finished_request(CURLM *multi) {
> struct CURLMsg *m;
> do {
> int msgq = 0;
> m = curl_multi_info_read(multi, &msgq);
> if(m && (m->msg == CURLMSG_DONE)) {
> CURL *e = m->easy_handle;
> curl_multi_remove_handle(multi, e);
> struct Request* request = lkup_request(e);
> close_request(request);
> }
> } while(m);
> }
>
> void deque_next_request(CURLM *multi) {
> // get url request and construct
> struct Request* request = alloc_request(url);
>
> curl_multi_add_handle(multi, request->easy);
> // add request to active queue
> }
>
> void thread_loop() {
> CURLM *multi = curl_multi_init();
> int still_running = 1;
>
> while(still_running) {
> deque_next_request();
> CURLMcode mc = curl_multi_perform(multi, &still_running);
>
> cleanup_finished_request();
> if(!mc && still_running)
>
> /* wait for activity, timeout or "nothing" */
> mc = curl_multi_poll(multi, NULL, 0, 1000, NULL);
>
> if(mc) {
> fprintf(stderr, "curl_multi_poll() failed, code %d.\n", (int)mc);
> break;
> }
> }
> }
>
> In alloc_request() create a request object which includes an easy handler and hold private data and install CURLOPT_CLOSESOCKETFUNCTION and CURLOPT_CLOSESOCKETDATA. The problem is that when the request is finished before the socket closes, the CLOSESOCKEDATA becomes invalid since it binds to the request object which will be free when the request finishes.
>
> I did some checks and the curl_easy_cleanup() did not reset the socket callback. It seems to me the close socket callback and its data only set once in allocate_conn() function in lib/url.c and won't be updated even if we recall this
> curl_easy_setopt(curl, CURLOPT_CLOSESOCKETFUNCTION, NULL);
> curl_easy_setopt(curl, CURLOPT_CLOSESOCKETDATA, NULL);
>
> Any recommendations/guidance for my use-case?

Not sure I fully understand what you are trying to do. For your understanding: easy handles use internal connection (which own the socket). There can be many easy handles using the same connection. With HTTP/1.1 there is only one at a time, but a connection can be used again for the next easy handle. With HTTP/2 you can have many (commonly up to 100) easy handles per connection.

So, when a transfer is done (easy cleanup), the connection lives on, the socket stays alive.

Does this explain to you what you are seeing?

- Stefan

>
> On Wed, May 29, 2024 at 2:53 PM Daniel Stenberg <daniel_at_haxx.se> wrote:
> On Wed, 29 May 2024, Cao Duc Quan via curl-library wrote:
>
> > We plan to add metrics to monitor connectivities such as socket open/close.
> > It seems to me that only CURL APIs support open/close socket callbacks
> > CURLOPT_CLOSESOCKETFUNCTION
> > <https://curl.se/libcurl/c/CURLOPT_CLOSESOCKETFUNCTION.html> but we do not
> > have similar APIs for CURLM.
>
> Right, because the multi handle has no sockets of its own really. Sockets are
> used for transfers and the transfers are held or owned by the easy handles.
>
> So, those are the open/close socket callbacks libcurl provides.
>
> > The problem is that the lifetime of the socket in CURLM maybe longer
> > compared with the easy handle, which means the CURL object could be finished
> > and removed before the socket is closed.
>
> Why is this a problem?
>
> --
>
> / daniel.haxx.se
> | Commercial curl support up to 24x7 is available!
> | Private help, bug fixes, support, ports, new features
> | https://curl.se/support.html
>
>
> --
> --------------------------------
> Watson Cao
> VN: (+84) 0976574864
> CA: (+1) 2368658864
> --
> Unsubscribe: https://lists.haxx.se/mailman/listinfo/curl-library
> Etiquette: https://curl.se/mail/etiquette.html

-- 
Unsubscribe: https://lists.haxx.se/mailman/listinfo/curl-library
Etiquette:   https://curl.se/mail/etiquette.html

Received on 2024-05-31