ssh command unexpectedly continues on other system after ssh terminates

uther's answer tells you to allocate a terminal but doesn't explain why. The reason is not specific to ssh, it is a matter of signal generation and propagation. I invite you to read What causes various signals to be sent? for more background.

On the remote host, there are two relevant processes:

• an instance of the ssh daemon (sshd), which is relaying the input and output of the remote program to the local terminal;
• a shell, which is running that for loop.

The shell can die either naturally when it reaches the end of the loop or experiences a fatal error, or to a signal. The question is, why would the shell receive a signal?

If the remote shell is connected to sshd over pipes, which is what happens when you specify a command on the ssh command line, then it will die of a SIGPIPE if sshd exits and the shell tries to write to the pipe. As long as the shell isn't writing to the pipe, it won't receive a SIGPIPE. Here, the shell is never writing anything to its standard output, so it can live forever.

You can pass the -t option to ssh, to tell it to emulate a terminal on the remote side and run the specified command in this terminal. Then, if the SSH client disappears, sshd closes the connection and exits, destroying the terminal. When the terminal device goes away, any process running in it receives a SIGHUP. So if you kill the SSH client (with kill, or by closing the terminal that the client is running in), the remote shell is SIGHUPped.

If you pass the -t option, SSH also relays SIGINT. If you press Ctrl+C, then the remote shell receives a SIGINT.

Try allocating a psuedo-tty with your ssh command.

ssh -t $ip_address 'for n in 1 2 3 4 5; do sleep 10; echo $n >>/tmp/count; done'

When you disconnect the ssh session, the process should terminate.

Since this is a pseudo-tty, your shell initialization is probably not going to source configuration files, leaving your command with a very bare environment. You may need to set up a .ssh/environment file that defines environment variables such as PATH. From man 1 ssh

Additionally, ssh reads ~/.ssh/environment, and adds lines of the format 
“VARNAME=value” to the environment if the file exists and users are allowed
to change their environment.  For more information, see the 
PermitUserEnvironment option in sshd_config(5).

As an alternative to using the -t option to ssh to make the remote command terminate when the ssh client exits or (gets killed), a named pipe can be used to convert "EOF to SIGHUP" when the stdin of sshd is being closed (see Bug 396 - sshd orphans processes when no pty allocated).

# sample code in Bash
# press ctrl-d for EOF
ssh localhost '
TUBE=/tmp/myfifo.fifo
rm -f "$TUBE"
mkfifo "$TUBE"
#exec 3<>"$TUBE"

<"$TUBE" sleep 100 &  appPID=$!

# cf. "OpenSSH and non-blocking mode", 
# http://lists.mindrot.org/pipermail/openssh-unix-dev/2005-July/023090.html
#cat >"$TUBE"
#socat -u STDIN "PIPE:$TUBE"
dd of="$TUBE" bs=1 2>/dev/null
#while IFS="" read -r -n 1 char; do printf '%s' "$char"; done > "$TUBE"

#kill -HUP -$appPID 
kill $appPID

rm -f "$TUBE"
'