The classic reference for this is this FAQ in the online Apache docs.
In that document, an
md5 of the modulus is used because
As the public exponent is usually 65537 and it's difficult to visually check that the long modulus numbers are the same, you can use the following approach
$ openssl rsa -noout -modulus -in key.pem.decrypted | openssl md5
$ openssl x509 -noout -modulus -in crt.pem | openssl md5
I'm proposing the use of process substitution to avoid visual comparison of the modulus:
$ diff \
<(openssl x509 -noout -modulus -in cert.pem) \
<(openssl rsa -noout -modulus -in key.pem.decrypted)
If the output is empty, the private key matches the certificate.
This is just a more convenient way to compare the strings than the one in Andrew Schulman's answer, which is, of course, also valid.
Another reference here.