ssh-add "Error connecting to agent: Connection refused"

Start the ssh-agent by running

$ eval "$(ssh-agent)"

in your terminal window.

I'm using iTerm2 on mac, I just Session -> Restart session