Re: Guidance on monitoring connection open/close with CURLM API

From: Cao Duc Quan via curl-library <curl-library_at_lists.haxx.se>
Date: Fri, 31 May 2024 07:57:33 -0700

Hi Stefan,
Thanks for the explanation.
To conclude:
- If the easy handle is not used with CURLM, the connection will be closed
when we call curl_easy_cleanup() so I could say connection lifetime is the
same as easy handle lifetime.
- If the easy handle is used with CURLM, the connection lifetime will be
longer than the easy handle. If we set the
CURLOPT_CLOSESOCKETFUNCTION/CURLOPT_CLOSESOCKETDATA
to easy handle, the callback could be invoked after the easy handle is
destroyed (afte invoke curl_easy_cleanup()).

On Thu, May 30, 2024 at 11:55 PM Stefan Eissing <stefan_at_eissing.org> wrote:

>
>
> > Am 31.05.2024 um 06:30 schrieb Cao Duc Quan via curl-library <
> curl-library_at_lists.haxx.se>:
> >
> > Hi,
> >
> > Here is the main flow in my application with CURLM
> >
> > struct Request {
> > CURL *easy;
> > // other data
> > }
> >
> > int on_close_socket(void* clientp, curl_socket_t sock) {
> > struct Request *request = (struct Request*)clientp;
> > // log some metrics for request
> > close(sock);
> > return 0;
> > }
> >
> > struct Request* alloc_request(const char* url) {
> > struct Request* request = malloc(sizeof(struct Request));
> >
> > request->easy = curl_easy_init();
> > curl_easy_setopt(curl, CURLOPT_CLOSESOCKETFUNCTION, on_close_socket);
> > curl_easy_setopt(curl, CURLOPT_CLOSESOCKETDATA, request);
> > // other init
> > }
> >
> > void close_request(struct Request* request) {
> > curl_easy_cleanup(request->easy);
> > free(request);
> > }
> >
> > struct Request* lkup_request(CURL *easy) {
> > // lookup request from easy in active queue
> > }
> >
> > void cleanup_finished_request(CURLM *multi) {
> > struct CURLMsg *m;
> > do {
> > int msgq = 0;
> > m = curl_multi_info_read(multi, &msgq);
> > if(m && (m->msg == CURLMSG_DONE)) {
> > CURL *e = m->easy_handle;
> > curl_multi_remove_handle(multi, e);
> > struct Request* request = lkup_request(e);
> > close_request(request);
> > }
> > } while(m);
> > }
> >
> > void deque_next_request(CURLM *multi) {
> > // get url request and construct
> > struct Request* request = alloc_request(url);
> >
> > curl_multi_add_handle(multi, request->easy);
> > // add request to active queue
> > }
> >
> > void thread_loop() {
> > CURLM *multi = curl_multi_init();
> > int still_running = 1;
> >
> > while(still_running) {
> > deque_next_request();
> > CURLMcode mc = curl_multi_perform(multi, &still_running);
> >
> > cleanup_finished_request();
> > if(!mc && still_running)
> >
> > /* wait for activity, timeout or "nothing" */
> > mc = curl_multi_poll(multi, NULL, 0, 1000, NULL);
> >
> > if(mc) {
> > fprintf(stderr, "curl_multi_poll() failed, code %d.\n", (int)mc);
> > break;
> > }
> > }
> > }
> >
> > In alloc_request() create a request object which includes an easy
> handler and hold private data and install CURLOPT_CLOSESOCKETFUNCTION and
> CURLOPT_CLOSESOCKETDATA. The problem is that when the request is finished
> before the socket closes, the CLOSESOCKEDATA becomes invalid since it binds
> to the request object which will be free when the request finishes.
> >
> > I did some checks and the curl_easy_cleanup() did not reset the socket
> callback. It seems to me the close socket callback and its data only set
> once in allocate_conn() function in lib/url.c and won't be updated even if
> we recall this
> > curl_easy_setopt(curl, CURLOPT_CLOSESOCKETFUNCTION, NULL);
> > curl_easy_setopt(curl, CURLOPT_CLOSESOCKETDATA, NULL);
> >
> > Any recommendations/guidance for my use-case?
>
> Not sure I fully understand what you are trying to do. For your
> understanding: easy handles use internal connection (which own the socket).
> There can be many easy handles using the same connection. With HTTP/1.1
> there is only one at a time, but a connection can be used again for the
> next easy handle. With HTTP/2 you can have many (commonly up to 100) easy
> handles per connection.
>
> So, when a transfer is done (easy cleanup), the connection lives on, the
> socket stays alive.
>
> Does this explain to you what you are seeing?
>
> - Stefan
>
> >
> > On Wed, May 29, 2024 at 2:53 PM Daniel Stenberg <daniel_at_haxx.se> wrote:
> > On Wed, 29 May 2024, Cao Duc Quan via curl-library wrote:
> >
> > > We plan to add metrics to monitor connectivities such as socket
> open/close.
> > > It seems to me that only CURL APIs support open/close socket callbacks
> > > CURLOPT_CLOSESOCKETFUNCTION
> > > <https://curl.se/libcurl/c/CURLOPT_CLOSESOCKETFUNCTION.html> but we
> do not
> > > have similar APIs for CURLM.
> >
> > Right, because the multi handle has no sockets of its own really.
> Sockets are
> > used for transfers and the transfers are held or owned by the easy
> handles.
> >
> > So, those are the open/close socket callbacks libcurl provides.
> >
> > > The problem is that the lifetime of the socket in CURLM maybe longer
> > > compared with the easy handle, which means the CURL object could be
> finished
> > > and removed before the socket is closed.
> >
> > Why is this a problem?
> >
> > --
> >
> > / daniel.haxx.se
> > | Commercial curl support up to 24x7 is available!
> > | Private help, bug fixes, support, ports, new features
> > | https://curl.se/support.html
> >
> >
> > --
> > --------------------------------
> > Watson Cao
> > VN: (+84) 0976574864
> > CA: (+1) 2368658864
> > --
> > Unsubscribe: https://lists.haxx.se/mailman/listinfo/curl-library
> > Etiquette: https://curl.se/mail/etiquette.html
>
>

-- 
--------------------------------
Watson Cao
VN: (+84) 0976574864
CA: (+1) 2368658864

-- 
Unsubscribe: https://lists.haxx.se/mailman/listinfo/curl-library
Etiquette:   https://curl.se/mail/etiquette.html

Received on 2024-05-31