Why is this seemingly valid argument using modus tollens not valid? How is this an example of "begging the question"?
It is not clear from the phrasing in your quote whether the statement "If $n$ is a real number with $n > 3$, then $n^2 > 9$" is meant to be a given premise or the statement that we're trying to prove.
If it's a premise, and we're trying to use it to prove that $n \le 3$, given the additional assumption that $n^2 \le 9$, then the argument is indeed correct: given $A \implies B$, we can deduce $\lnot A$ from $\lnot B$.
If, instead, the first statement in the quoted argument is supposed to be the theorem that we're trying to prove, then the proof is invalid. While we could indeed deduce $n > 3 \implies n^2 > 9$ from $n^2 \le 9 \implies n \le 3$ (those two statements being logically equivalent), the quoted argument contains nothing that could be used to prove $n^2 \le 9 \implies n \le 3$ in the first place, unless we incorrectly assume the statement that we're trying to prove (or something else that has not been explicitly stated).
Presumably, the book you're reading explains how such arguments given in it are to be parsed. In particular, if the arguments in the book are written in the format "Theorem to be proved: Proof of theorem," then this particular argument is invalid; if they're instead meant to be parsed as "Established premises: Assumptions. Result," then it's valid.
Unfortunately, while there exist several established conventions like this, none are quite universal enough to be safely assumed without knowing which convention the author follows. When writing proofs yourself, to avoid such confusion it's usually a good idea to clearly and explicitly indicate which statements are axioms or prior theorems, which are temporary assumptions, and which are the new theorems that you're trying to prove.
It's not really a proof, that's the problem. It's just rephrasing the statement and then claiming that it's obvious. If the arguer actually showed that if $n^2 \leq 9$ then $n \leq 3$, it would be a valid proof by contraposition.
It's like saying: "We prove Fermat's Last Theorem. Indeed, let $n \geq 3$. Then $x^n + y^n = z^n$ is equivalent to $y^n + x^n = z^n$, which never happens, so we're done.".
I think this may just be a (mutual) misunderstanding between you and the textbook of the symbol ":" in the actual textbook question.
The textbook meaning of ":" seems to be "Because".
Taking that as the ":" meaning, the argument is:
If n is a real number with n>3, then n^2>9 (because,) if n^2≤9, then n≤3.
So, if the second part (if n^2≤9, then n≤3
) is supposed to be an argument in the favor of the implication n>3==>n^2>9
, it actually is not, as it is just re-stating (begging) the question.
While you (and, mind you, I as well) would read ":" as "therefore", making it a legitimate modus tollens:
If n is a real number with n>3, then n^2>9 (therefore,) if n^2≤9, then n≤3.
Edit: My apologies for not noticing - this is actually what @Ilmari Karonen wrote in his earlier answer.