Buy commercial curl support from WolfSSL. We help you work
out your issues, debug your libcurl applications, use the API, port to new
platforms, add new features and more. With a team lead by the curl founder
himself.
Re: How to handle CA certificate bundles in portable application bundles (e.g., AppImages)?
- Contemporary messages sorted: [ by date ] [ by thread ] [ by subject ] [ by author ] [ by messages with attachments ]
From: Timothe Litt <litt_at_acm.org>
Date: Thu, 19 May 2022 20:31:34 -0400
> Can you query the path to the system CA certificate from libcurl
> reliably (e.g., using some dedicated function)?
I don't think so. curl-config --configure may have "--with-ca-bundle=",
but if the bundle was defaulted by configure, I don't think it will be
listed.
Several limitations around the default bundle path have been noted:
It would be nice to have a curl_easy_getinfo() along the lines of:
|curl_easy_getinfo(CURL *curl, CURLINFO CURLINFO_CAINFO, char **path)|
to get a string with the current bundle path for a handle, and with the
'curl' argument NULL, the built-in library default. Neither is
currently accessible.
This would also make it possible to reset a curl handle to the default
bundle path, another recent request.
Optionally, having |curl_easy_setopt( curl, CURLOPT_CAINFO, NULL)| would
be a handy shortcut for
|char *default_bundle = NULL; CURLcode err;||
|
|if( (err = curl_easy_getopt( NULL, CURLINFO_CAINFO, &default_bundle))
== CURLE_OK ) ) {|
| err = curl_easy_setopt( NULL, CURLOPT_CAINFO, default_bundle ); /*
Or use a CURL handle instead of NULL to reset it */||
|
| free( default_bundle ); /* maybe */
|
| default_bundle = NULL;|
|} return err;||
|
> As far as I can see, there are both files bundles and directory
> bundles, which would have to be concatenated.
I don't see an efficient way to merge the (CAPATH) directories
containing certificates. These contain softlinks with the hash of
certificates pointing to an individual certificate file. The point is
to avoid reading all the certificates - when one is needed, the library
uses the hash to read just the one. This is only a win if there are a
LOT of certificates.
Neither libcurl nor the application knows what certs will be needed in
advance. So if you want to search several directories, you'd be left
with either reading all the certs in all the directories into a big
bundle file, or creating a temporary directory and copying all the links
from directories in your search list into the temporary directory. Then
specify that as a CURLOPT_CAPATH. Either way would be so expensive as
to defeat the optimization.
A middle ground would be to specify the larger directory as a CAPATH,
and merge the certs in the smaller into a big CAINFO bundle. This would
require application knowledge, and still wouldn't be very efficient.
As with CAINFO bundle merging, to make the cost worthwhile, the result
would have to be reused, and that will be more efficient if done externally.
Note that only a subset of TLS libraries support the CAPATH mechanism....
For all these reasons, I think this is even lower priority than merging
CAINFO bundles...
Timothe Litt
ACM Distinguished Engineer
--------------------------
This communication may not represent the ACM or my employer's views,
if any, on the matters discussed.
On 19-May-22 06:07, TheAssassin via curl-library wrote:
> On 19.05.22 11:43, Timothe Litt via curl-library wrote> On 19-May-22
> 04:38, Daniel Stenberg via curl-library wrote:
>>> Supporting more than one file is an interesting idea. This current
>>> single file restriction (except for OpenSSL which can have a
>>> directory of certs) is primarily because how the TLS libraries' APIs
>>> for doing certificate loading/verification. It could certainly be
>>> changed to support multiple files, but it would not be an
>>> inisignificant amount of work...
>>>
>> Although I suppose one could come up with some use cases, isn't this
>> exactly equivalent to concatenating the multiple files into one? If
>> it's not convenient to do this externally to the application, this is
>> easily done by the application with a tempfile - more easily than by
>> libcurl due to lifetime considerations.
>
> Such a crypto lib API limitation is something I was concerned would
> exist, and obviously requires such workarounds to support this use
> case. I agree that it makes sense to handle this inside the
> applications then, maybe with some support of libcurl by means of
> dedicated helper functions.
>
> Can you query the path to the system CA certificate from libcurl
> reliably (e.g., using some dedicated function)? This would help a lot
> while adding this to an application. Ideally, the list generated by
> CURL_CHECK_CA_BUNDLE could be fetched as well. Both could be available
> from curl-config if a runtime function is too much overhead, but I
> think a function within libcurl would be the better solution.
>
> By the way, some use cases of such a feature:
>
> - an application wants to extend the list with a self-signed
> certificate (e.g., to allow access to a private service which you
> cannot easily get a certificate for)
> - one wants to test some piece of code against a local server, but
> needs to make requests to third-party services on the Internet in the
> process (for instance, an OIDC login from a local test server using an
> external IdP)
>
>> Such a temp file will go away at exit, or if bigbundle is closed.
>> (knowing when to close it is what might be tricky for libcurl,
>> depending on when the TLS library is done with the file.)
>>
>> If the list is used more than once, creating the big bundle will be
>> more efficient if done externally - even if libcurl tried to cache
>> it, which would get messy.
>
> True words. If it is not possible for libcurl to forward a list, then
> it's easier to move this process into the application, which passes
> the result libcurl for every request, and handles cleanup. An
> application could even cache on the filesystem this result if needed,
> for instance.
>
>> If there were enough use cases to justify it, I guess concatenating
>> files into a tempfile could be a libcurl convenience function... it
>> doesn't seem like a a lot of work either way.
>
> As far as I can see, there are both files bundles and directory
> bundles, which would have to be concatenated. Creating a helper in
> libcurl would be beneficial to users so that they do not have to
> reimplement such code over and over again.
>
Received on 2022-05-20
Date: Thu, 19 May 2022 20:31:34 -0400
> Can you query the path to the system CA certificate from libcurl
> reliably (e.g., using some dedicated function)?
I don't think so. curl-config --configure may have "--with-ca-bundle=",
but if the bundle was defaulted by configure, I don't think it will be
listed.
Several limitations around the default bundle path have been noted:
It would be nice to have a curl_easy_getinfo() along the lines of:
|curl_easy_getinfo(CURL *curl, CURLINFO CURLINFO_CAINFO, char **path)|
to get a string with the current bundle path for a handle, and with the
'curl' argument NULL, the built-in library default. Neither is
currently accessible.
This would also make it possible to reset a curl handle to the default
bundle path, another recent request.
Optionally, having |curl_easy_setopt( curl, CURLOPT_CAINFO, NULL)| would
be a handy shortcut for
|char *default_bundle = NULL; CURLcode err;||
|
|if( (err = curl_easy_getopt( NULL, CURLINFO_CAINFO, &default_bundle))
== CURLE_OK ) ) {|
| err = curl_easy_setopt( NULL, CURLOPT_CAINFO, default_bundle ); /*
Or use a CURL handle instead of NULL to reset it */||
|
| free( default_bundle ); /* maybe */
|
| default_bundle = NULL;|
|} return err;||
|
> As far as I can see, there are both files bundles and directory
> bundles, which would have to be concatenated.
I don't see an efficient way to merge the (CAPATH) directories
containing certificates. These contain softlinks with the hash of
certificates pointing to an individual certificate file. The point is
to avoid reading all the certificates - when one is needed, the library
uses the hash to read just the one. This is only a win if there are a
LOT of certificates.
Neither libcurl nor the application knows what certs will be needed in
advance. So if you want to search several directories, you'd be left
with either reading all the certs in all the directories into a big
bundle file, or creating a temporary directory and copying all the links
from directories in your search list into the temporary directory. Then
specify that as a CURLOPT_CAPATH. Either way would be so expensive as
to defeat the optimization.
A middle ground would be to specify the larger directory as a CAPATH,
and merge the certs in the smaller into a big CAINFO bundle. This would
require application knowledge, and still wouldn't be very efficient.
As with CAINFO bundle merging, to make the cost worthwhile, the result
would have to be reused, and that will be more efficient if done externally.
Note that only a subset of TLS libraries support the CAPATH mechanism....
For all these reasons, I think this is even lower priority than merging
CAINFO bundles...
Timothe Litt
ACM Distinguished Engineer
--------------------------
This communication may not represent the ACM or my employer's views,
if any, on the matters discussed.
On 19-May-22 06:07, TheAssassin via curl-library wrote:
> On 19.05.22 11:43, Timothe Litt via curl-library wrote> On 19-May-22
> 04:38, Daniel Stenberg via curl-library wrote:
>>> Supporting more than one file is an interesting idea. This current
>>> single file restriction (except for OpenSSL which can have a
>>> directory of certs) is primarily because how the TLS libraries' APIs
>>> for doing certificate loading/verification. It could certainly be
>>> changed to support multiple files, but it would not be an
>>> inisignificant amount of work...
>>>
>> Although I suppose one could come up with some use cases, isn't this
>> exactly equivalent to concatenating the multiple files into one? If
>> it's not convenient to do this externally to the application, this is
>> easily done by the application with a tempfile - more easily than by
>> libcurl due to lifetime considerations.
>
> Such a crypto lib API limitation is something I was concerned would
> exist, and obviously requires such workarounds to support this use
> case. I agree that it makes sense to handle this inside the
> applications then, maybe with some support of libcurl by means of
> dedicated helper functions.
>
> Can you query the path to the system CA certificate from libcurl
> reliably (e.g., using some dedicated function)? This would help a lot
> while adding this to an application. Ideally, the list generated by
> CURL_CHECK_CA_BUNDLE could be fetched as well. Both could be available
> from curl-config if a runtime function is too much overhead, but I
> think a function within libcurl would be the better solution.
>
> By the way, some use cases of such a feature:
>
> - an application wants to extend the list with a self-signed
> certificate (e.g., to allow access to a private service which you
> cannot easily get a certificate for)
> - one wants to test some piece of code against a local server, but
> needs to make requests to third-party services on the Internet in the
> process (for instance, an OIDC login from a local test server using an
> external IdP)
>
>> Such a temp file will go away at exit, or if bigbundle is closed.
>> (knowing when to close it is what might be tricky for libcurl,
>> depending on when the TLS library is done with the file.)
>>
>> If the list is used more than once, creating the big bundle will be
>> more efficient if done externally - even if libcurl tried to cache
>> it, which would get messy.
>
> True words. If it is not possible for libcurl to forward a list, then
> it's easier to move this process into the application, which passes
> the result libcurl for every request, and handles cleanup. An
> application could even cache on the filesystem this result if needed,
> for instance.
>
>> If there were enough use cases to justify it, I guess concatenating
>> files into a tempfile could be a libcurl convenience function... it
>> doesn't seem like a a lot of work either way.
>
> As far as I can see, there are both files bundles and directory
> bundles, which would have to be concatenated. Creating a helper in
> libcurl would be beneficial to users so that they do not have to
> reimplement such code over and over again.
>
-- Unsubscribe: https://lists.haxx.se/listinfo/curl-library Etiquette: https://curl.haxx.se/mail/etiquette.html
- application/pgp-signature attachment: OpenPGP digital signature