Can RDP clients launch remote applications and not desktops

Using an RDP connection file you can set the alternate shell to be your application; the file syntax is like

alternate shell:s:c:\winnt\system32\notepad.exe

and you pass that as a command-line argument to mstsc.exe; this similar to chrissr's solution, but without affecting every RDP session you launch. A fuller summary of settings here.

"alternate shell" doesn't seem to work anymore in recent versions of Windows, RemoteApp is the way to go.

remoteapplicationmode:i:1
remoteapplicationname:s:Purpose of the app shown to user...
remoteapplicationprogram:s:C:\...\some.exe
remoteapplicationcmdline:s:

To get this to work under e.g. Windows 10 Professional, one needs to enable some policy:

[HKEY_LOCAL_MACHINE\SOFTWARE\Policies\Microsoft\Windows NT\Terminal Services]
"fAllowUnlistedRemotePrograms"=dword:00000001