SSL error with a newly signed cert?

Last night I literally spent hours figuring out an alleged issue with the certificate from StartCom. Of course the problem was entirely on my end, in the editor to be precise. But what happened?

I fetched ca-bundle.pem and entered it as ssl.ca-file. Furthermore I concatenated my private key used for the CSR and the signed cert I got from StartCom (excellent service in every respect) into a PEM file that I assigned in lighttpd using the ssl.pemfile directive. Then I tried to restart the server (shortened output for brevity):

# service lighttpd restart
Stopping web server: lighttpd.
Starting web server: lighttpd [...] (network.c.607) SSL: Private key does not match the certificate public key, [...]
 failed!

Wait! But I had just gotten the cert from the StartCom control panel, pasted it into my PEM file and did the same with the key.

Inspecting the certificate public key modulus and comparing it with the one from the private key brought a surprise:

# openssl rsa -modulus -noout -in domain.pem
unable to load Private Key
16986:error:0906D06C:PEM routines:PEM_read_bio:no start line:pem_lib.c:650:Expecting: ANY PRIVATE KEY

… uhm, that is essentially what lighttpd was telling me already. I looked at the old working PEM for another domain and saw no obvious differences there. So I decided to exchange the key and certificate positions and retry:

# openssl x509 -modulus -noout -in domain.pem
unable to load certificate
17095:error:0906D06C:PEM routines:PEM_read_bio:no start line:pem_lib.c:650:Expecting: TRUSTED CERTIFICATE

… I thought I’m onto something here.

Eventually I was sanity-checking some assumptions that the inspection inside Vim and my other editor on Windows seemed to support. Alright:

# grep '^-----' domain.pem
-----END RSA PRIVATE KEY-----
-----BEGIN CERTIFICATE-----
-----END CERTIFICATE-----

… opening the file in an editor again seemed to disprove the silly grep(1) output, until it dawned on me.

Of course, when I pasted the cert had created a new file. My editor was set to default to UTF-8 and thus must have prepended the BOM (byte order marker) to the file. However, every self-respecting editor is going to suppress that and instead show you some subtle piece of information in the status bar or so, telling you of the fact. Sure enough file(1) agreed with me:

# file key.pem
domain.pem: UTF-8 Unicode (with BOM) text

Removing the BOM was relatively easy (did it on the stored keys and certs, of course), but I wanted to verify upfront what file(1) would say. So I did:

# tail -c +4 key.pem|file -
/dev/stdin: PEM RSA private key

Fair enough. So I removed it on the actual file:

# tail -c +4 key.pem > key.pem
$ file key.pem
key.pem: PEM RSA private key

End of story. It works now:

# grep '^-----' domain.pem
-----BEGIN RSA PRIVATE KEY-----
-----END RSA PRIVATE KEY-----
-----BEGIN CERTIFICATE-----
-----END CERTIFICATE-----

// Oliver

PS: thanks to Eddy Nigg from StartCom for some pointers and questioning some of my assumptions.