EC2 instance always ask me to enter passphrase for the pem during connection

Most likely your test.pem is corrupt.

To check if it is fine run openssl rsa -check -in test.pem -noout

It should say "RSA key ok"