Is it possible to change the default value of $profile to a new value?

The only thing I can think of is "dot sourcing" your profile at the powershell invocation.

For example:

powershell -noprofile -noexit -command "invoke-expression '. ''C:\My profile location\profile.ps1''' "

By changing the script that invoke-expression command points to you can place your "profile" anywhere you'd like. Then, create shortcut that launches PowerShell and set the target to the above command.

You can change your $Profile.CurrentUser* paths by changing your personal folder path Environment.GetFolderPath(Environment.SpecialFolder.Personal)

Either via regedit

HKEY_CURRENT_USER\Software\Microsoft\Windows\CurrentVersion\Explorer\User Shell Folders

Under the Name column select Personal and chage the value to where you want your profile.

Or via PowerShell

New-ItemProperty 'HKCU:\SOFTWARE\Microsoft\Windows\CurrentVersion\Explorer\User Shell Folders' Personal -Value 'Your New Path Here' -Type ExpandString -Force

You have to reboot for this to take effect.

You can also put your profile file here

C:\Windows\System32\WindowsPowerShell\v1.0\profile.ps1

If you want to have a separated location for all your profiles and scripts, you can modify your profile.ps1 file above as

 $profile = "NewLocation\profile.ps1"
. $profile

Make sure what type of profile you use, see details here

https://technet.microsoft.com/en-ca/library/dd819434.aspx