There are two points to consider when verifying a certificate:
- verifying it has been emitted by an entity you trust, and
- verifying it matches the identity of the server you're trying to contact.
The Public Key Infrastructure using X.509 cert. specification defines the structure to put in place for this. It's a hierarchical model where you get a tree, the root of which are Certification Authorities (CA) and its leafs are the end-entity certificates (in particular server certificates).
The root CA certificates tend to be self-signed. A bunch of them are included by default in your OS or browser. That's the "leap of faith" part, where you trust your OS/browser vendor to have vetted the CAs to do their job properly. Some of the big commercial CAs are Verisign, Thawte, ...
A CA may then sign other certificates. You can verify a certificate you've never seen before by checking whether it has been signed by a CA you trust. There may also be intermediate CAs.
For example, the certificate for
https://www.google.com/ has been signed by "Thawte SGC CA", which has itself been signed by "VeriSign, Inc./Class 3 Public Primary Certification Authority" (I'm shorting the names here to simplify).
The job of the CA is to verify (by means external to the PKI) that the person/institution to which it issues the certificate is the legitimate owner of that host name (e.g.
www.google.com here). The way this out-of-band verification varies, for non-EV certificates, this is often done simply by sending an e-mail to the address that registered the domain name (available in the whois directory).
Once this is done, suppose you don't know whether to trust
https://www.google.com/, the client verify this certificate against the trust anchors it has (the CAs, often pre-imported by OS/browser vendors). If you connect to
www.google.com, the browser gets the certificate and is then able to work out the chain up to the top issuer (there's an issuer entry in every certificate), until it finds one it trusts (in this case the one from Verisign). So far, so good, the certificate is trusted. The second step consists of checking that this certificate is indeed for this machine. For HTTPS, this is done by checking that the hostname is in the subject alternative name extension of the certificate, or as a fallback, that it's in the "Common Name" (CN) entry in the subject distinguished name. This is explained in RFC 2818 (server identity).
Here the possible attempts to attack are as follows:
- The attackers forges a certificate (for that hostname or not), with their own CA. Since their CA isn't recognised by your browser the certificate won't be verified. Even if they faked the issuer name, they wouldn't be able to fake the signature (because you verify using the public key using the CA certificates you already trust). One of the biggest problems here was in the strength of the signature: collision attacks have been demonstrated using the MD5 digest algorithm, so CAs now use SHA-1 instead, which is considered more robust. If you consider RSAwithSHA1 or DSAwithSHA1 to be sufficiently robust, you shouldn't have a problem there.
- The attackers get a legitimate certificate from a well-known CA, but for a different host name (as a CA shouldn't emit to someone else). Let's say they get
www.example.com. You're trying to connect to
www.google.com, they redirect the traffic to their box, which will show a certificate that will be verifiable by a CA you trust, but for
www.example.com. This is where the host name verification is important. Your browser will warn you that you're not connected to the host you intended.
This system is designed to ensure that, if a MITM redirects the traffic to their machine, the client will not accept the connection. This is of course only valid if the user doesn't ignore the warnings shown by the browser.
It's the same principle, except that you're the CA, and this self-signed cert may also directly be the server certificate. If you create your own self-signed certificate and put it on your machine, you can also import it in your browser as a trusted authority, in which case you follow the same procedure as described above.
Some browsers, like Firefox, will let you add permanent exceptions to these rules. If you know, by some other means (e.g. the admin gave you in person the certificate), what the certificate for the machine you want to connect to is, you can choose to trust them explicitly, even if they haven't been signed by a CA you trust or if the name doesn't match. Of course, for this, you do need to know a priori and explicitly what this particular certificate should be.
If in either cases, the user chooses to ignore the warnings and accepts to be redirected to a connection with an untrusted certificate, then the MITM (with that untrusted certificate) can see/redirect/alter the traffic.