Why is SSH password authentication a security risk?
Solution 1:
Using ssh keys do have one unique feature compared to password login: you can specify the allowed commands. This can be done by modifying ~/.ssh/authorized_keys file at the server.
For example,
command="/usr/local/bin/your_backup_script.sh", ssh-rsa auiosfSAFfAFDFJL1234214DFAfDFa...
would allow only the command `/usr/local/bin/your_backup_script.sh" with that particular key.
You can also specify the allowed hosts for the key:
from="yourclient,yourotherclient", ssh-rsa auiosfSAFfAFDFJL1234214DFAfDFa...
Or combine the two:
from="yourbackupserver", command="/usr/local/bin/your_backup_script.sh", ssh-rsa auiosfSAFfAFDFJL1234214DFAfDFa...
With keys you can also grant temporary access to some user (say, a consultant) to a server without revealing the password for that particular account. After the consultant has finished his/her job, the temporary key can be removed.
Solution 2:
You can get the best of both worlds by only allowing password authentication from within your network. Add the following to the end of your sshd_config:
PasswordAuthentication no
Match Address 10.0.0.0/8,172.16.0.0/12,192.168.0.0/16
PasswordAuthentication yes
Solution 3:
There are pro's and con's for either pw or key-based authentication.
In some cases, for example, key-based authentication is less secure than password authentication. In other cases, its pw-based that's less secure. In some cases, one is more convenient, in others, less.
It all boils down to this: When you do key-based authentication, you must secure your key with a passphrase. Unless you have ssh-agent running (ssh-agent frees you from entering your passphrase every time), you've gained nothing in terms of convenience. Security is disputable: the attack vector now shifted from the server to YOU, or your account, or your personal machine, (...) - those may or may not be easier to break.
Think outside of the box when deciding this. Whether you gain or loose in terms of security depends on the rest of your environment and other measures.
edit: Oh, just saw that you're talking about a home server. I was in the same situation, "password" or "USB stick with key on it" always with me? I went for the former but changed the SSH listening port to something different than 22. That stops all those lame script kiddies brute forcing whole network ranges.
Solution 4:
You have partially responded to your question - the more places attacker can connect from, the more possibilities he has to break into your server by bruteforcing (consider DDoS).
Also compare the length of your password with the key size (usually thousands of bits).
Solution 5:
When you log in with a password you transmit your password to the server. This means that the operator of the server can modify the SSHD to get access to your password. With public key authentication, they cannot obtain your private key as only your public key every goes to the server.