Curl doesn't recognize certificates in PEM format in keychai

% curl -V
curl 7.21.4 (universal-apple-darwin11.0) libcurl/7.21.4 OpenSSL/0.9.8r zlib/1.2.5
Protocols: dict file ftp ftps gopher http https imap imaps ldap ldaps pop3 pop3s rtsp smtp smtps telnet tftp
Features: AsynchDNS GSS-Negotiate IPv6 Largefile NTLM SSL libz

here's some output from curl that tripped me up initially:

% curl -I https://some.serv.ver -v
* About to connect() to some.serv.ver port 443 (#0)
* Trying some.ip... connected
* Connected to some.ser.ver (some.ip) port 443 (#0)
* SSLv3, TLS handshake, Client hello (1):
* SSLv3, TLS handshake, Server hello (2):
* SSLv3, TLS handshake, CERT (11):
* SSLv3, TLS alert, Server hello (2):
* SSL certificate problem, verify that the CA cert is OK. Details:
error:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed
* Closing connection #0
curl: (60) SSL certificate problem, verify that the CA cert is OK. Details:
error:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed
More details here: http://curl.haxx.se/docs/sslcerts.html

curl performs SSL certificate verification by default, using a "bundle"
of Certificate Authority (CA) public keys (CA certs). If the default
bundle file isn't adequate, you can specify an alternate file
using the --cacert option.
If this HTTPS server uses a certificate signed by a CA represented in
the bundle, the certificate verification probably failed due to a
problem with the certificate (it might be expired, or the name might
not match the domain name in the URL).
If you'd like to turn off curl's verification of the certificate, use
the -k (or --insecure) option.

I then tried to reproduce this from the command line without the keychain being involved and got a similar error on the command line:

% curl -I https://some.ser.ver -vvv --cacert ~/Desktop/db.insideguidance.com.der
* About to connect() to some.ser.ver port 443 (#0)
* Trying some.ip... connected
* Connected to some.server (some.ip) port 443 (#0)
* error setting certificate verify locations:
CAfile: /path/to/some.ser.ver.der
CApath: none

* Closing connection #0
curl: (77) error setting certificate verify locations:
CAfile: /path/to/some.ser.ver.der
CApath: none

I think this is due to how OpenSSL works. I can't spot any particular code flow for DER...

Just a wild guess, but could this be due to PEM being the default format and you having to do extra work to support other formats as well? (I.e. openssl not having the default to try all three parsers till it finds the one that works?)

Or are you saying that curl doesn't support the other certificate formats at all?

again: I think this is due to how OpenSSL works. I don't see us doing any particular different actions depending on the format. If you think otherwise, please be more specific.

I don't know the code so I can't really comment on that.

But to get this to closure if possible: what can I do to help fix this bug? Do you think this bug is with OpenSSL rather than with curl? If so, how could we verify that? How could I help with that? Can you replicate the issue?

I think it is either a bug or a (known?) limitation in OpenSSL, yes.

How to verify? I don't know SSL nor OpenSSL good enough to tell. You probably need to dive in some source code and see what happens or not and figure out the reasoning. Or perhaps ask a friendly OpenSSL developers to see if she/he can shed some light onto this.

I haven't even tried to replicate the problem. I haven't really had the time or energy to do it, not to mention that I'm not even sure exactly what command line or test case I would do to repeat it. I don't have your cert.

If you need it I can invest the time to re-create the situation with a new certificate so that I can upload it here (really just a self signed root certificate, a server with a certificate signed with that).

As for OpenSSL developers, I really don't know any, if you can direct me to a the right bug tracker for problems like this I'd be delighted to help bring them in. I would guess however that you have far better connections to them, so I would prefer to direct my energies to making it easy for you guys to reproduce this issue.

If you can provide a test case I'll promise to give it a shot, although I cannot guarantee any particular time frame nor can I guarantee any outcome.

This set of events seems to reliably reproduce the problem on my machine:

Can you reproduce the problem from that?

-- snip --
# Generate private key
% openssl genrsa -des3 -out server_private_key.rsa.pem

# Create certificate signing request
% openssl req -new -key server_private_key.rsa.pem -out server_certificate_signing_request.csr
Common Name (eg, YOUR name) []:localhost

# Decrypt private key (so we don't need a password each time it is used
% openssl rsa -in server_private_key.rsa.pem -out server_private_key.pem

# Create server certificate
% openssl x509 -req -days 365 -in server_certificate_signing_request.csr -signkey server_private_key.pem -out server_certificate.pem

# Serve http using that certificate
% openssl s_server -cert server_certificate.pem -key server_private_key.pem -www

# Test it
% curl -i https://localhost:4433 --cacert server_certificate.pem

# Convert certificate
% openssl x509 -in server_certificate.pem -out server_certificate.der -outform DER

# Test it again
% curl -i https://localhost:4433 --cacert server_certificate.der
curl: (77) error setting certificate verify locations:
CAfile: server_certificate.der
CApath: none

Added as known bug #80. Bug entry closed.