Export push notification certificate as P12
You can't export to a .p12 because you don't have the private key installed on your machine (if you did, you would have an "expand" arrow next to the certificate like your iPhone Distribution certificate has.) You will need to get the private key from the machine that generated the certificate before you export.
The solution I have found has been posted here: https://stackoverflow.com/a/19502944/1198404
I copy and paste the answer: Turns out all you have to do is select "My Certificates" on the left panel and it enables the .p12 option.