How to set Gmail profile picture for third party emails?

A method that should work, though I'm unsure that this is the right site for this (possibly SuperUser?), is creating a Google+ account with that e-mail address and then adding a profile picture to it.

If you are looking to avoid a Google account, this may not be the best approach, but it's somewhat unavoidable if you want to interact with their system in this way.

After Google closed down G+ for non-G-Suite users the solution given before no longer works. An effective way is now setting up your webmail as an alternate email in your google account settings as explained here