git remote add with other SSH port

You need to edit your ~/.ssh/config file. Add something like the following:

Host example.com
    Port 1234

A quick google search shows a few different resources that explain it in more detail than me.

You can just do this:

git remote add origin ssh://user@host:1234/srv/git/example

1234 is the ssh port being used