reCAPTCHA:"ERROR for site owner: Invalid site key"

It wasn't obvious to me initially but you have to supply your site key in 2 places:

You need to ensure your site is added in the domains section of the google reCAPTCHA page where you generated the API key and save. Then test again. That worked for me.

In my case it was using a wordpress site,

And I had chosen version 3 of captcha and it turns out the wordpress only supported version 2.