Buy commercial curl support. We
help you work out your issues, debug your libcurl applications, use the API,
port to new platforms, add new features and more. With a team lead by the
curl founder Daniel himself.
Re: Asynchronous certificate verification and curl_multi
- Contemporary messages sorted: [ by date ] [ by thread ] [ by subject ] [ by author ] [ by messages with attachments ]
From: Stefan Eissing via curl-library <curl-library_at_lists.haxx.se>
Date: Thu, 14 Aug 2025 16:42:45 +0200
> Am 14.08.2025 um 11:25 schrieb Ondra via curl-library <curl-library_at_lists.haxx.se>:
>
> Here is a proof of concept which currently only works with curl_multi_socket_action() API so far: https://github.com/curl/curl/pull/18284
>
> -after asynchronous cert verification is detected, the easy prevents fds from being scheduled
> -when cert verification finishes, the user needs to first call
> curl_easy_setopt(easy CURLOPT_ASYNC_CERT_VERIFY_FINISHED, 1L);
> and subsequently call
> curl_multi_socket_action(multi, CURL_SOCKET_RETRY_CERT);
>
> It works, but it is pretty crude and ugly; I would be grateful for guidance regarding which directions to take. The interface changes seem far from ideal, and in the implementation, especially use of data->conn->bits.cert_verification_* from cf-socket.c looks like a hack, and keeping the easy in multi->process during this time seems unclean...
>
My proposal to address this https://github.com/curl/curl/pull/18288
> On Wed, Aug 13, 2025 at 9:44 AM Daniel Stenberg <daniel_at_haxx.se> wrote:
> On Wed, 13 Aug 2025, Ondra via curl-library wrote:
>
> > According to https://curl.se/libcurl/c/CURLOPT_SSL_CTX_FUNCTION.html, “For
> > OpenSSL, asynchronous certificate verification via *SSL_set_retry_verify* is
> > supported. (Added in 8.3.0 <https://curl.se/ch/8.3.0.html>)”, so I have
> > attempted to use this approach; but I have found out, that although when I
> > register my callback via SSL_CTX_set_cert_verify_callback and then
> > successfully call SSL_set_retry_verify from it before returning from it
> > indeed ensures that next time the easy handle is processed by curl_multi,
> > the verification callback is called again, curl does not seem to do anything
> > more.
>
> I think this is simply because this is not widely used nor understood
> functionality. Looking at it now, I think it was unfortunate that we accepted
> the change like that. Underdocumented and no test case that verifies it.
>
> Maybe it is broken? Maybe it needs something else to work.
>
> > The issue is that while the asynchronous verification is pending, we need
> > for the easy handle to be inert, and when verification finishes, we need to
> > resume processing of that handle. Currently it seems that the easy handle
> > stays in previous state with regards of sockets scheduled on it’s behalf by
> > multi into the eventloop; since this socket can be (and in some cases
> > actually is) triggered (eg. readable) for the whole time, this will lead to
> > busy looping (as the cert verify callback is invoked again and again and we
> > have to use SSL_set_retry_verify every time).
>
> The question is really what else curl should do. I mean, if it doesn't do that
> then when should it call the callback again? I'm afraid I don't know how this
> asynch cert thing works in OpenSSL so I can't really tell.
>
> > Please what is the proper way how to handle this issue?
>
> I think it is pretty clear that this is unknown (to most of us). I think you
> need to figure that out and possibly provide a patch that makes curl work
> better with this. A test case and an example code would be excellent and bring
> bonus points.
>
> --
>
> / daniel.haxx.se || https://rock-solid.curl.dev
> --
> Unsubscribe: https://lists.haxx.se/mailman/listinfo/curl-library
> Etiquette: https://curl.se/mail/etiquette.html
Date: Thu, 14 Aug 2025 16:42:45 +0200
> Am 14.08.2025 um 11:25 schrieb Ondra via curl-library <curl-library_at_lists.haxx.se>:
>
> Here is a proof of concept which currently only works with curl_multi_socket_action() API so far: https://github.com/curl/curl/pull/18284
>
> -after asynchronous cert verification is detected, the easy prevents fds from being scheduled
> -when cert verification finishes, the user needs to first call
> curl_easy_setopt(easy CURLOPT_ASYNC_CERT_VERIFY_FINISHED, 1L);
> and subsequently call
> curl_multi_socket_action(multi, CURL_SOCKET_RETRY_CERT);
>
> It works, but it is pretty crude and ugly; I would be grateful for guidance regarding which directions to take. The interface changes seem far from ideal, and in the implementation, especially use of data->conn->bits.cert_verification_* from cf-socket.c looks like a hack, and keeping the easy in multi->process during this time seems unclean...
>
My proposal to address this https://github.com/curl/curl/pull/18288
> On Wed, Aug 13, 2025 at 9:44 AM Daniel Stenberg <daniel_at_haxx.se> wrote:
> On Wed, 13 Aug 2025, Ondra via curl-library wrote:
>
> > According to https://curl.se/libcurl/c/CURLOPT_SSL_CTX_FUNCTION.html, “For
> > OpenSSL, asynchronous certificate verification via *SSL_set_retry_verify* is
> > supported. (Added in 8.3.0 <https://curl.se/ch/8.3.0.html>)”, so I have
> > attempted to use this approach; but I have found out, that although when I
> > register my callback via SSL_CTX_set_cert_verify_callback and then
> > successfully call SSL_set_retry_verify from it before returning from it
> > indeed ensures that next time the easy handle is processed by curl_multi,
> > the verification callback is called again, curl does not seem to do anything
> > more.
>
> I think this is simply because this is not widely used nor understood
> functionality. Looking at it now, I think it was unfortunate that we accepted
> the change like that. Underdocumented and no test case that verifies it.
>
> Maybe it is broken? Maybe it needs something else to work.
>
> > The issue is that while the asynchronous verification is pending, we need
> > for the easy handle to be inert, and when verification finishes, we need to
> > resume processing of that handle. Currently it seems that the easy handle
> > stays in previous state with regards of sockets scheduled on it’s behalf by
> > multi into the eventloop; since this socket can be (and in some cases
> > actually is) triggered (eg. readable) for the whole time, this will lead to
> > busy looping (as the cert verify callback is invoked again and again and we
> > have to use SSL_set_retry_verify every time).
>
> The question is really what else curl should do. I mean, if it doesn't do that
> then when should it call the callback again? I'm afraid I don't know how this
> asynch cert thing works in OpenSSL so I can't really tell.
>
> > Please what is the proper way how to handle this issue?
>
> I think it is pretty clear that this is unknown (to most of us). I think you
> need to figure that out and possibly provide a patch that makes curl work
> better with this. A test case and an example code would be excellent and bring
> bonus points.
>
> --
>
> / daniel.haxx.se || https://rock-solid.curl.dev
> --
> Unsubscribe: https://lists.haxx.se/mailman/listinfo/curl-library
> Etiquette: https://curl.se/mail/etiquette.html
-- Unsubscribe: https://lists.haxx.se/mailman/listinfo/curl-library Etiquette: https://curl.se/mail/etiquette.htmlReceived on 2025-08-14