Apple - How to send encrypted email using Mail app?

This is how it works here, with the help of a self-signed root certificate:

Step 1: Alice creates a self-signed root certificate

• In the Keychain app, select Keychain Access > Certificate Assistent > Create a Certificate Authority
• Select
  • Identity Type: Self Signed Root CA
  • User Certificate: S/MIME (Email)
  • Let me override defaults: yes
  • Email from: Enter an email address for the root CA (could be Alice's)
  • On the next dialog, select Sign your invitation: no - optionally create a CA web site
  • On the next dialog, enter whatever CA data you want
  • On the next two dialogs, leave Key Pair Information as is
  • On the next dialog Key Usage Extension
• Include Key Usage Extension: yes
• This extension is critical: no
• Select: Signature, Certificate Signing (optionally more, but not required for a root certificate)
• On the next dialog, leave Key Usage Extension for the user's certificate as is
• On the next dialog, optionally set Include Extended Key Usage Extension = yes, if you want to use the root CA for more than just email certificates - but always leave This extension is critical = no
• On the next dialog, leave Use Basic Constraints Extension = Use this certificate as a certificate authority
• On the remaining dialogs, leave selections as they are
• On the last dialog Specify a Location For The Certificate, you can select System, and also select On this machine, trust certificates signed by this CA = yes

Step 2: Alice and Bob trust the the root CA

• If Alice hasn't already trusted the CA certificate during the previous step, she now selects the new root CA certificate in her Keychain app and explicitly trusts the CA certificate
• Alice exports the public CA certificate in .cer format and publishes it to Bob
• Bob imports the public certificate and explicitly trusts this CA

Since Alice and Bob have explicitly trusted the CA, all certificates signed by this CA will be automatically trusted on their computers.

Step 3: Alice creates an email certificate for herself.

• In the Keychain app, select Keychain Access > Certificate Assistent > Create a Certificate
• Enter Alice's name, then select
  • Identity Type: Leaf
  • User Certificate: S/MIME (Email)
  • Let me override defaults: yes
• Leave next dialog (serial no. etc.) as is
• On the next dialog, for Email Address, enter Alice's email address this certificate is intended for, optionally fill out remaining fields as desired
• On the next dialog Choose An Issuer, select the CA created in step 1
• On the next dialog, keep Key Pair Information as is
• On the next dialog Key Usage Extension
• Include Key Usage Extension: yes
• This extension is critical: yes
• Select: Signature, Key Encipherment (optionally, adding Data Encipherment seems to be possible, but do not add more!)
• • On the next dialog, leave Key Usage Extension for the user's certificate as is
• On the next dialog, optionally set Include Extended Key Usage Extension = yes, but set This extension is critical = no
  • Select "Email Protection*, add more if you want to use the same certificate for other purposes
• On the next dialog, leave Include Basic Constraints Extension = no
• On the next dialog, select Include Subject Alternate Name Extension = yes
  • Leave This extension is critical = no
  • Enter the email address (again) in the rfc822Name field. Optionally add more email addresses, separated by space, which you may want to use the certificate for
• On the last dialog Specify a Location For The Certificate, you can select Login

Step 4: Alice creates a certificate for Bob

Now Alice repeats step 3 to create a certificate for Bob, then exports his certificate in .p12 format, gives it to him, and Bob imports it into his keychain.

Note: This is the easiest way to create Bob's certificate, yet not the recommended way. This is because Alice creates Bob's private key, so Bob has to trust Alice. Actually Bob would use the Certificate Assistent on his computer to Request a Certificate From a Certificate Authority.

Step 5: Alice and Bob send each other signed messages

Alice sends a signed email message to Bob, and Bob sends a signed email message to Alice. If all went well, the signatures will display as trusted because both trust the CA certificate.

Step 6: Alice and Bob can exchange encrypted email messages

Both can reply to the signed message using encryption with the reply message.