how do I replace a string with a dollar sign in it in powershell

Unless you modify your original string (e.g. by escaping the $), this is isn't (really) possible. Your $string doesn't really contain a $PSP, as it is replaced by nothing in the assignment statement.

$string = "this is a sample of 'my' text $PSP.what do you think"
$string -eq "this is a sample of 'my' text .what do you think"

evaluates to:

True

Use the backtick character (above the tab key):

$string = "this is a sample of 'my' text `$PSP.what do you think"

To replace the dollar sign using the -replace operator, escape it with backslash:

"this is a sample of 'my' text `$PSP.what do you think" -replace '\$PSP', 'hello'

Or use the string.replace method:

$string = "this is a sample of 'my' text `$PSP.what do you think"
$string.Replace('$PSP','Hello)'

this is a sample of 'my' text Hello.what do you think