Duplicity restore failing: No secret key

The problem is, like the linked post stated, that gpg 2.1 retires passphrase from pipe for key auth.
The gpg agents needs to be enabled and configured for the restore to work.

Add the following to ~/.gnupg/gpg.conf:

use-agent
pinentry-mode loopback

And to your ~/.gnupg/gpg-agent.conf:

pinentry-program /usr/bin/pinentry-gtk-2
allow-loopback-pinentry

Then restart the agent with echo RELOADAGENT | gpg-connect-agent.

The restore work even if the keys are only on the local machine. I still don't get why it does not ask for the passphrase when making incremental though.