What is this operator called?
This operation ${\rm Ops}(4)$ is called tetration, from the greek root tetra meaning four; it's also sometimes called a "power tower". There are also many further generalizations of this type of sequence; Knuth's up-arrow notation gives $a^{a^{a^a}}=a\uparrow\uparrow4$, so that $a\uparrow\uparrow n$ is the tetration operation. By adding more arrows you get pentation and so on, and the Conway chained arrow notation generalizes this still further.
FYI, for "to the power of-ation" the word you're looking for is exponentiation.
A more general function that combines all those operators has been defined by Ackermann:
$ \varphi(m,n,p) = \begin{cases} \varphi(m, n, 0) = m + n \\ \varphi(m, 0, 1) = 0 \\ \varphi(m, 0, 2) = 1 \\ \varphi(m, 0, p) = m &\text{ for } p > 2 \\ \varphi(m, n, p) = \varphi(m, \varphi(m, n-1, p), p - 1) &\text{ for } n > 0 \text{ and } p > 0. \end{cases} $
So for $p = 0, 1, 2$ you get
$\phi(m, n, 0) = m + n $
$\phi(m, n, 1) = m \cdot n $
$\phi(m, n, 2) = m ^ n $
and
$\phi(m, n, 3) = \overbrace{{{m ^ m} ^ m} ^ {...}}^{n}$
It is known as a tetration, and it is normally written as $^na$ where n is the height of the power tower. It is the forth hyperoperation.
The zeroth hyperoperation is the successor function, and the first is the zeroth hyperoperation iterated, and so on
A more general way to define the nth hyperoperation is, using the notation, $H_n(a,b)$ where n is the nth hyperoperation,
${\displaystyle H_{n}(a,b)={\begin{cases}b+1&{\text{if }}n=0\\a&{\text{if }}n=1{\text{ and }}b=0\\0&{\text{if }}n=2{\text{ and }}b=0\\1&{\text{if }}n\geq 3{\text{ and }}b=0\\H_{n-1}(a,H_{n}(a,b-1))&{\text{if }n\in\mathbb{N},n>3}\end{cases}}}$
Some notations for hyperoperations are(for $H_n(a,b)$:
- Square bracket notation: $a[n]b$
- Box notation: $a{\,{\begin{array}{|c|}\hline {\!n\!}\\\hline \end{array}}\,}b$
- Nambiar's notation : $a\otimes ^{n-1}b$
- Knuth's up arrow notation: $a\uparrow^{n-2}b$
- Goodstien's notation: $G(a,b,n)$
- Conway's chained arrow notation: $a\rightarrow b\rightarrow (n-2)$
- Bowers exploding array function: $\{a,b,n,1\}$
- Original Ackermann function: ${\begin{matrix}\phi (a,b,n-1)\ {\text{ for }}1\leq n\leq 3\\\phi (a,b-1,n-1)\ {\text{ for }}n\geq 4\end{matrix}}$