Git always asks for my ssh-key passphrase

Like Nasreddine says, it's because your key is encrypted with a passphrase to prevent others from reading it.

The most convenient way of using passphrased keys is by using ssh-agent to start an authentication agent (which runs in the background):

$ eval "$(ssh-agent)"
Agent pid 44304

...and then using ssh-add to register your key with the agent so that it's used automatically for subsequent SSH connections:

$ ssh-add ~/.ssh/id_rsa
Enter passphrase for /home/khanhpn/.ssh/id_rsa:

You'll be asked to enter your passphrase when you run ssh-add, but you won't need to do so again while ssh-agent is running.

You can find more information on GitHub. Also note that you'll have to do this every time you log in, so you might like to add the eval "$(ssh-agent)" step to your .profile script.

It's because your private SSH key is protected using a passphrase. You can remove it using this command (not recommended since anyone could copy your key and use it to access your repos/account):

$ ssh-keygen -p
Enter file in which the key is (/path/to/your/.ssh/key):

enter your current passphrase when prompted here:

Enter old passphrase:
Key has comment 'rsa w/o comment'

leave empty if you want to remove the passphrase when prompted here:

Enter new passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved with the new passphrase.