Sv: Re: Curl callback question, porting from OpenSSL 1.x to 3.x and from 32bit plain to 64bit UTF16

From: Anders Gustafsson via curl-library <curl-library_at_lists.haxx.se>
Date: Mon, 25 Aug 2025 08:50:48 +0300

I guess if we leave the, however valid, points on string handling aside I guess what I really wanted to ask
was about the difference between the sample code, ie:

X509_STORE_add_cert(store, cert)

Where the cert was created from a PEM format string

and

SSL_CTX_use_certificate((SSL_CTX*)sslctx, cert);
SSL_CTX_use_PrivateKey((SSL_CTX*)sslctx, pkey);

where cert and key are created the same way through a BIO

I guess this is not even a CURL question per se, but rather an OpenSSL one?

Ie: https://docs.openssl.org/3.0/man3/X509_STORE_add_cert

So I guess I should ask in the OpenSSL forums?

-- 
Med vänlig hälsning
Anders Gustafsson, ingenjör
anders.gustafsson_at_pedago.fi  |  Support +358 18 12060  |  Direkt +358 9 315 45 121  |  Mobil +358 40506 7099
Pedago interaktiv ab, Nygatan 7 B , AX-22100 MARIEHAMN, ÅLAND, FINLAND
>>> Andreas Mohr <andi_at_lisas.de> 2025-08-20 17:44 >>>
Hi,
disclaimer:
quite experienced in certain areas, yet not too much in others (CURL).
TL;DR:
thus discussing potential string encoding issues only.
On Wed, Aug 20, 2025 at 03:58:14PM +0300, Anders Gustafsson via curl-library wrote:
> So, yes this is windows 😀 libcurl/8.15.0-DEV OpenSSL/3.5.2 zlib/1.3.1
> 
> I had some issues and I just want to check whether I am going about this the right way. The function calls
an
> API where the client certificate is used to authenticate the caller so in the original version I used the
> sslctx_function(). To complicate matters does my app support PEM certificates and keys in two different
ways:
> 1. As files (Say on a removable secure media) and 2. As strings in the database for ease of use.
> 
> The first way (filenames) worked right away, ie:
> 
> 			m_Certificate.Trim();
> 			if (m_Certificate.IsEmpty())
> 				curl_easy_setopt(curl, CURLOPT_SSLCERT, m_CertificateFile.GetString());
Bleeep - ATLMFC CStringT::GetString() encountered.
This might thus be
dirty "encoding shortcutting" here
(simply invoking .GetString() to
"quickly" "get at" some "char"-compatibly-typed - hah! - input, rather than
doing active transcoding to
the actually *correct* encoding spec of
some char-typed handling).
Thus, consulting this one:
> Where m_Certificate and m_Key and regular (char) strings with the PEM coded data.
What would "regular" mean?
Considering that
CString errfilename;
with
CT2A(errfilename),
one would think that this is
a CString[T] with UNICODE config setting (put differently, CStringW)
environment, however
since you said "regular (char) strings", I am assuming that
you are on !UNICODE config (i.e., CStringA).
https://manpages.ubuntu.com/manpages/kinetic/man3/CURLOPT_SSLCERT.3.html 
Yeah nice - that page does not specify at all which
encoding char *cert (a filesystem item argument!! - which could be
containing all the smileys available in this universe, with
only a bit of luck...) is expected to have.
Thus on Windows one would tend to
assume ACP crap - which of course would mean that it is
Unicode-compliance-broken (since: neither UTF16 nor UTF8 nor UTF7 nor
UTF-EBCDIC or whatever ;-)).
> 	errno_t fileerr = fopen_s(&errfile, CT2A(errfilename), "w+, ccs=UTF-8");
Unicode-compliance-broken filesystem item handling!
CT2A(errfilename) will be
wide-typed to CP_ACP transition (in UNICODE config setting), and
"nothing" *) (in non-UNICODE).
*) BTW *HORRIBLE* atlconv.h comment "// Code page doesn't matter" atlconv.h transcoding protocol breakage!!!
Yeah, as if that would be the case for
e.g. CP_ACP to CP_UTF8 transcoding, which *is* a valid transcoding use case...)
(think of
CT2CA(..., CP_UTF8)
protocol behaviour **DIFFERENCE**)
Thus, your errfilename possibly is ACP (CP_ACP, GetACP()) content,
which would be
"compatible" since
fopen_s() API is equally ACP-specced on Windows (yuck).
...but: then it would be
Unicode-compliance-broken (due to
being ACP crap, rather than
e.g. UTF8 as usually on Linux).
Since fopen_s() should have an overload for wide-typed input I'd think,
the way to go would at least be
CT2W(errfilename) - thereby
properly preserving Unicode-compliant (since wide-typed!) encoding (when
on UNICODE config setting - and ACP crap on !UNICODE).
Or, better do utf8everywhere.org (i.e., std::string[-means-utf8] -
to have ensured that
**every** string traffic anywhere is Unicode-compliant), and thus do
std::string errfilename = "myUtf8InputStuffStringFromSomewhere"; // e.g. std::filesystem **) API
fopen_s(... CA2W(errfilename, CP_UTF8) ...);
**) rather *horribly* Unicode-compliance-broken (on Windows!) - I digress...
"<filesystem>: prevent filesystem::path dangerous conversions to/from default code page encoding"
  https://github.com/microsoft/STL/issues/909 
> In the second scenario, PEM in database, I had some problems and I just wanted to check that the code I
came
> up with is sane. Ie the authentication will not happen unless I have both certificate and key, so:
> 
> 			if (!m_Certificate.IsEmpty())
> 			{
> 				curl_easy_setopt(curl, CURLOPT_SSL_CTX_FUNCTION, sslctx_function);
> 				curl_easy_setopt(curl, CURLOPT_SSL_CTX_DATA, m_Certificate.GetString());
WARNING CORRUPTION: CURLOPT_SSL_CTX_DATA has a void*-typed ptr arg, thus
both .GetString() CString[T] is accepted, *silently*).
IOW, once on UNICODE config setting it would be
*broken*.
So, questionable encoding stuff again.
According to
https://curl.se/libcurl/c/CURLOPT_SSL_CTX_DATA.html 
  "char *mypem = /* CA cert in PEM format"
it seems to *appear* that
"PEM format" means some plain ASCII-only payload.
Now to be maximally precise one could do
const UINT nCP_PEM = 20127 /*CP_ASCII*/ /* these clearly are [to be!] all ASCII-only, right!!!? */;
std::string strCertificate = CW2A(CA2W(m_Certificate), nCP_PEM);
(this transition expects that m_Certificate has system ACP content, of course)
(OTOH one could just assume that
all [relevant] ACP encodings are ASCII subset, thus
simply NOT do transcoding since it then ought to be
ASCII-compliant content already anyway).
> Then, below, which seems to work OK. I first used the example here:
> https://curl.se/libcurl/c/CURLOPT_SSL_CTX_FUNCTION.html 
> but that one did not fix my key for me. Yes, this code still leaves allocated memory in case of errors.
"fix my key" - that wording might hint at
encoding issues.
But perhaps we're talking about
a plain CURL certificate config issue only after all.
I could not precisely identify (thus: discuss) particular ***) issues in
your handling, but I'd hope that
this will give you some ideas (*if* it is an encoding issue).
***) well, except for the broken fopen_s() filesystem item handling...
Greetings
Andreas Mohr
-- 
Unsubscribe: https://lists.haxx.se/mailman/listinfo/curl-library
Etiquette:   https://curl.se/mail/etiquette.html

Received on 2025-08-25