Can I restrict a Certification Authority to signing certain domains only?

Thomas Pornin's answer is good, but a little outdated. Support for Name Constraints is growing.

I've found that OpenSSL 1.0.1k and Windows 7 support the extension.

Test

Using XCA, I created a self-signed CA certificate, and added a critical Name Constraints extension for .lab.example.com, by adding the following line on the "Advanced" tab during certificate creation:

nameConstraints=critical,permitted;DNS:.lab.example.com

Note: the constraint should not have a leading dot. It's technically incorrect, but support for this is expanding: https://github.com/golang/go/commit/e4dafa32620e80e4e39937d8e2033fb2ee6085f8

Then, I used that CA certificate to sign two other certificates for HTTPS servers:

• test.lab.example.com - Valid
• bad.google.com - Clearly invalid

Next, after setting up DNS entries accordingly, I used this modified simple-https-server.py to run an HTTPS server, once with each of the generated certificates:

./simple-https-server --certfile test.lab.example.com.pem --hostname test.lab.example.com

and

./simple-https-server --certfile bad.google.com.pem --hostname bad.google.com

After installing the CA certificate into the OS trust, I then tried to visit each site with several clients.

Results

OpenSSL 1.0.1k seems to support this. curl gave me the following error when I tried to visit bad.google.com:

curl: (60) The Certifying Authority for this certificate is not permitted to issue a certificate with this name.

Chrome on Windows 7 also does the right thing. Chrome gives a fairly generic net::ERR_CERT_INVALID, but Windows certificate viewer is quite explicit:

The certificate has an invalid name. The name is not included in the permitted list or is explicitly excluded.

References

• http://karl.kornel.us/2014/09/cas-name-constraints-and-a-business-opportunity

---

Update 1

I also tried signing a certificate that did not specify a Subject Alternative Name, instead relying on the old common-name only.

OpenSSL / curl still refused to accept the certificate.

Both Chrome and IE11 on Windows refused to accept the certificate on Windows, even though windows itself (when viewing the server certificate) didn't complain about it. To me, that means that the browsers are doing more than simply asking the OS to verify the certificate, which is a good thing.

---

Note that name constraints are not properly supported on OSX earlier than 10.13.3 which was released early 2018.

---

Conclusion

I feel secure in asking others to install my root CA certificate, without putting them at any risk.

No.

(I assume you are talking about certificates for SSL servers.)

Technically no. What would be closest to that would be the Name Constraints extension (see section 4.2.1.10 of RFC 5280) (OID 2.5.29.30), which theoretically allows for restricting a complete PKI subtree to an explicit set of domains (and subdomains thereof). The extension supports both whitelist and blacklist semantics (in your case, you would like a whitelist). In practice, however, this fails for two reasons:

• The Name Constraints extension is mostly unsupported by existing implementations of SSL. They are likely to ignore the extension.

• When a SSL client connects to a server, it looks for the server name in the server certificate, as specified in RFC 2818, section 3.1. It will look for names of type dNSName in a Subject Alt Name extension, and these names are covered (theoretically) by the Name Constraints. However, if the server certificate lacks a Subject Alt Name extension, clients will fall back on the Common Name (in the subjectDN). The Common Name is not in scope of the Name Constraints. This means that a certificate could evade the name constraints by omitting the Subject Alt Name extension and putting an arbitrary server name in its Common Name.

(This is the whole story of X.509: lots of hooks and provisions for many useful features, which don't work because of lack of support from implementation and lack of coordination between specification bodies.)