This article was originally published on Scott Helme’s blog and is reprinted here with his permission.
We have a little problem on the web right now and I can only see it becoming a larger concern as time goes by: more and more sites are obtaining certificates, vitally important documents needed to deploy HTTPS, but we have no way of protecting ourselves when things go wrong.
Certificates
We’re currently seeing a bit of a gold rush for certificates on the Web as more and more sites deploy HTTPS. Beyond the obvious security and privacy benefits of HTTPS, there are quite a few reasons you might want to consider moving to a secure connection that I outline in my article Still think you don’t need HTTPS?. Commonly referred to as “SSL certificates” or “HTTPS certificates”, the wider Internet is obtaining them at a rate we’ve never seen before in the history of the web. Every day I crawl the top one million sites on the Web and analyze various aspects of their security and every 6 months I publish a report. You can see the reports here, but the main result to focus on right now is the adoption of HTTPS.
Not only are we continuing to deploy HTTPS, the rate at which we’re doing so is increasing, too. This is what real progress looks like. The process of obtaining a certificate has become more and more simple over time and now, thanks to the amazing Let’s Encrypt, it’s also free to get them. Put simply, we send a Certificate Signing Request (CSR) to the Certificate Authority (CA) and the CA will challenge us to prove our ownership of the domain. This is usually done by setting a DNS TXT record or hosting a challenge code somewhere on a random path on our domain. Once this challenge has been satisfied the CA it issues the certificate and we can then present it to visitors’ browsers and get the green padlock and “HTTPS” in the address bar.
I have a few tutorials to help you out with this process, including how to get started, how to set up smart renewal, and how to use dual certificates. So, this is all great. What’s the problem?
The problem is when things don’t go according to plan and you have a bad day.
“We’ve been hacked!”
Nobody ever wants to hear those words, but the sad reality is that we do—more often than any of us would like. Hackers can go after any number of things when they gain access to our servers and often one of the things they can access is our private key. The certificates we use for HTTPS are public documents—we send them to anyone that connects to our site—but the thing that stops other people using our certificate is that they don’t have our private key. When a browser establishes a secure connection to a site, it checks that the server has the private key for the certificate it’s trying to use, and this is why no one but us can use our certificate. If an attacker gets our private key, though, things change.
Now that an attacker has managed to obtain our private key, they can use our certificate to prove that they are us. Let’s say that again: if your key is stolen, that means there is somebody on the Internet who is not you, who can prove that they are you. This is a real problem, and before you think “this will never happen to me,” you should recall Heartbleed. This tiny bug in the OpenSSL library allowed attackers to steal private keys and you didn’t even have to do anything wrong for it to happen. On top of this there are countless ways that private keys are exposed by accident or negligence. The simple truth is that we can lose our private keys, and when this happens, we need a way to stop an attacker from using our certificate. We need to revoke the certificate.
Revocation
In a compromise scenario we revoke our certificate so that an attacker can’t abuse it. Once a certificate is marked as “revoked,” Web browsers will know not to trust it, even though the certificate is valid. The owner has requested revocation and no client should accept it.
Once we know we’ve had a compromise, we contact the CA and ask that they revoke our certificate. We need to prove ownership of the certificate in question, and once we do that, the CA will mark the certificate as revoked. Now that the certificate is revoked, we need a way of communicating this revocation to any client that might require the information. Right after the revocation, visitors’ browsers doesn’t know about it—and of course, that’s a problem. There are two mechanisms that we can use to make this information available: a Certificate Revocation List (CRL), or the Online Certificate Status Protocol (OCSP).
Certificate Revocation Lists
A CRL is a really simple concept and is quite literally just a list of all certificates that a CA has marked as revoked. A client can contact the CRL Server and download a copy of the list. Armed with a copy of the list the browser can check to see if the certificate it has been provided is on that list. If the certificate is on the list, the browser now knows the certificate is bad and it shouldn’t be trusted. The browser should throw an error and abandon the connection. If the certificate isn’t on the list then everything is fine and the browser can continue the connection.
The problem with a CRL is that they contain a lot of revoked certificates from the particular CA maintaining it. Without getting into too much detail, they are broken down by each intermediate certificate a CA has and the CA can fragment the lists into smaller chunks. Regardless of how it’s broken up, the point I want to make remains the same: the CRL is typically not an insignificant size. The other problem is that if the client doesn’t have a fresh copy of the CRL, it has to fetch one during the initial connection to your site—which can make your site look much slower than they actually are.
This doesn’t sound particularly great—so how about we take a look at OCSP?
Online Certificate Status Protocol
The OCSP provides a much nicer solution to the problem and has a significant advantage over the CRL approach. With OCSP, we ask the CA for the status of a single, particular certificate. This means all the CA has to do is respond with a good/revoked answer, which is considerably smaller than a CRL. Great stuff!
It is true that OCSP offers a significant performance advantage over fetching a CRL, but, that performance advantage comes with a cost (don’t you hate it when that happens?). The cost is a pretty significant one, too: your privacy.
When we think about what an OCSP request is—the request for the status of a very particular, single certificate—you may start to realize that you’re leaking some information. When you send an OCSP request, you’re basically asking the CA this:
Is the certificate for pornhub.com valid?
So, not exactly an ideal scenario. You’re now advertising your browsing history to some third party that you didn’t even know about, all in the name of HTTPS—which set out to give us more security and privacy. Kind of ironic, huh?
Hard fail
But wait: there’s something else. I talked about the CRL and OCSP responses above, the two mechanisms a browser can use to check if a certificate is revoked. They look like this.
Upon receiving the certificate, the browser will reach out to one of these services and perform the necessary query to ultimately ascertain the status of the certificate. But what if your CA is having a bad day and the infrastructure is offline? What if it looks like this?
The browser has only two choices here. It can refuse to accept the certificate because it can’t check the revocation status, or it can take a risk and accept the certificate without knowing the revocation status. Both of these options come with their advantages and disadvantages. If the browser refuses to accept the certificate, then every time your CA has a bad day and their infrastructure goes offline, your sites goes offline, too. If the browser continues and accepts the certificate then it risks using a certificate that could have been stolen, and exposes the user to the associated risks thereof.
It’s a tough call—but right now, today, neither of these actually happen.