With ssh, how can you run a command on the remote machine without exiting?

You can try the following command to start up bash and source your own profile when you ssh:

ssh  -t hostname "bash --rcfile ~user/.bashrc"

Building on dogbane's answer, a complete solution looks like:

ssh -t user@server 'cd /a/great/place; bash'

Here I use -t to force the allocation of a pseudo-terminal, which is required for an interactive shell. Then I execute two commands on the server: first the thing I wanted to do prior to opening the interactive shell (in my case, changing directory to a specific folder), and then the interactive shell itself. bash sees that it has a pseudo-terminal and responds interactively.

The single quotes ensure the entire thing is passed to the remote server as the command to be run by your default shell.

Thanks again to dogbane for providing the necessary clue with -t. I used to solve this problem with expect, which is definitely killing a mouse with a cannon. (:

Try this solution.

echo "command" | ssh user@remote_host

The login is interactive and your command is passed as if you typed it on the interactive command line. The session terminates as if you had typed

ssh user@remote_host 'command'