How to run one command with a directory as argument, then cd to the same? I get "no such file or directory"

Instead of defining a function, you can use the variable $_, which is expanded to the last argument of the previous command by bash. So use:

cd "$_"

after mv command.

You can use history expansion too:

cd !:$

If you must use a function:

follow () { cd "$_" ;}

---

$ follow () { cd "$_" ;}
$ mv foo.sh 'foo bar'
$ follow 
foo bar$ 

N.B: This answer is targeted to the exact command line arguments format you have used as we are dealing with positional parameters. For other formats e.g. mv -t foo bar.txt, you need to incorporate specific checkings beforehand, a wrapper would be appropriate then.

With standard bash keybindings, the combination Alt. will copy the last argument of the previous command line into the current one. So, typing

$ mv foo ~/some/long/path/
$ cd <Alt><.>

would yield

$ mv foo ~/some/long/path/
$ cd ~/some/long/path/

and would be even less typing than the word follow.

For added convenience, repeating the Alt. combination will browse through the last arguments of all previous command lines.

Addendum: The bash command name corresponding to this key combination is yank-last-arg or insert-last-argument. It can be found in the bash manpage under "Commands for Manipulating the History" or in the more exhaustive Bash Reference Manual.)

You're almost certainly running into the problem that tilde expansion takes place before parameter expansion, which can be explained by a succinct example:

$ cd ~kaz
kaz $ var='~kaz'
kaz $ echo $var
~kaz
kaz $ cd $kaz
bash: cd: ~kaz: No such file or directory

This can be addressed with eval. Anyway, you're going to need eval, because you're pulling commands from the history and they can contain arbitrary expansions, like:

$ mv file.tex ~/Documents/$(compute_folder_name foo-param)/subfolder1
$ follow

(There are issues, such as that the re-expansion of these values might no longer match the original expansion which occurred. Suppose compute_folder_name is a function which increments some global variable.)