SSH Keychain for Xfce

If you've installed Seahorse (sudo apt-get install seahorse), add export $(gnome-keyring-daemon --daemonize --start) to your .bashrc and you'll get the same SSH key behaviour as in vanilla Ubuntu.

I think this is what you're looking for:

Setup of SSH agent in Xubuntu 11.10 to get password-less authentication with use of public key | Hnygard.no

If you have not already done so, set up your private and public key (See Githubs description about SSH key half way through Set Up Git). It is important that you add a passphrase to your private key.

Install the package ssh-askpass:

sudo apt-get install ssh-askpass

Add «/usr/bin/ssh-add» to your start up. Go to Settings – Settings manager – Session and startup. Go to Application Autostart and add an item that runs the command «/usr/bin/ssh-add«:

On next login you’ll get the following dialog asking for your passphrase. After entering it, you can open a terminal or use sftp in the file manager without typing a password.

Use ssh-agent and ssh-add?