When you buy a certificate, does the certificate authority have (a copy of) the private key?

Depending on how the CA does things, it may or may not have a copy of your private key. Usually it doesn't.

The normal method is that you generate your private/public key pair on your own machine, then send the public key to the CA as part of a certificate request. The CA assembles and signs the certificate, and sends it back to you. Your private key never left your machine, and the CA never saw it.

However, in some cases, it is a good idea to let the CA generate the key pair, and send it to you. One situation where this is desirable is for asymmetric encryption keys: if you lose a private key, then you lose all the data which has been encrypted with the corresponding public key, since you can no longer decrypt it. Therefore, encryption private keys should be backupped somewhere, and having the CA generate the private key makes it easy for the CA to enforce a comprehensive, inescapable backup system.

To know what happens with a specific CA, have a look at what they return to you: if the CA sends to you a PFX (PKCS#12) archive, then CA has the private key, or at least had the private key (whether it saved it is another matter). On the other hand, if the CA sends to you a raw certificate only, then it does not have the private key.

In any case, the whole process should be documented by the CA with which you are doing business; if the CA does not document what it does, and in particular where private keys are generated and whether they are stored or not, then find another CA. CA which do things opaquely cannot be trusted. Indeed, it is a basic requirements of all "CA best practice guides" (e.g. Web Trust) that a CA must document everything.

When the CA signs the locally generated keypair, the private key is never seen by the CA server.

If you use IE in a corporate environment, or a similarly configured EXE, it may be possible possible for that website to request the ActiveX control to upload the private key to the CA for key archival/escrow purposes.