Finding the derivative of $x \mapsto x^n$ for large $n$ using the definition

The expression you give is the derivative of the function $x \mapsto x^{123}$. There are a few ways that you can attack this, however nearly all of them are ultimately going to rest on an induction argument somewhere along the line. The other answers cite the binomial theorem, which is usually proved by induction. I'll give another argument, which does not rely on the binomial theorem.

First, let us recall the principle of mathematical induction. The basic idea is that if we want to show that some proposition $P$ holds for all natural numbers, then we (1) prove that $P(1)$ holds, then (2) prove that if $P(k)$ happens to hold for some $k$, then it must be that $P(k+1)$ holds. This is sufficient to prove that $P(n)$ holds for all $n$.

The intuition is as follows: suppose that we have shown that $P(1)$ holds and that $P(k) \implies P(k+1)$. Then, with $k=1$, we have that $P(2) = P(1+1)$ holds. But now $P(2)$ holds, and so with $k=2$, we have that $P(3) = P(2+1)$ holds. But now $P(3)$ holds, and so...

The page to which I linked above describes this as setting up a sequence of dominoes, which is a nice analogy.

With respect to the current problem, we wish to show that $$ \frac{\mathrm{d}}{\mathrm{d}x} x^n = n x^{n-1} $$ for all $n \in \mathbb{N}$. To do this, I am going to first assume that a basic theory of limits has been developed for functions on $\mathbb{R}$. With the basic arithmetic of limits assumed, we first prove a little lemma:

Lemma (The Product Rule): Let $f$ and $g$ be differentiable functions. Then $$ \frac{\mathrm{d}}{\mathrm{d}x} \left[ f(x) g(x) \right] = \left[ \frac{\mathrm{d}}{\mathrm{d}x} f(x) \right] g(x) + f(x) \left[ \frac{\mathrm{d}}{\mathrm{d}x} g(x) \right]. $$ That is, $(fg)' = f'g + fg'$.

Proof: By definition of the derivative \begin{align} \frac{\mathrm{d}}{\mathrm{d}x} \left[ f(x) g(x) \right] &= \lim_{h\to 0} \frac{f(x+h)g(x+h) - f(x)g(x)}{h} \\ &= \lim_{h\to 0} \frac{f(x+h)g(x+h) - f(x)g(x+h) + f(x)g(x+h) - f(x)g(x)}{h} \tag{1} \\ &= \lim_{h\to 0} \left[ \frac{f(x+h)-f(x)}{h}g(x+h) + f(x) \frac{g(x+h) - g(x)}{h} \right] \\ &= \left[ \lim_{h\to 0} \frac{f(x+h)-f(x)}{h} \right] \left[ \lim_{h\to 0} g(x+h) \right] + f(x) \left[ \lim_{h\to 0} \frac{g(x+h)-g(x)}{h} \right] \tag{2} \\ &= \left[ \frac{\mathrm{d}}{\mathrm{d}x} f(x) \right] g(x) + f(x) \left[ \frac{\mathrm{d}}{\mathrm{d}x} g(x) \right]. \end{align} At (1), we are just adding zero in a clever way. At (2), we use the fact that addition and multiplication are continuous. In elementary calculus classes, these properties are usually taught as specific rules to be memorized, e.g. "the limit of a sum is the sum of the limits." Finally, the last line follows from the definition of the derivative and the fact that if $g$ is differentiable, then it is also continuous, and so $\lim_{h\to 0} g(x+h) = g(x)$. //

We can now prove the desired result:

Theorem (The Baby Power Rule): Let $n \in \mathbb{N} = \{1,2,\dotsc\}$. Then $$ \frac{\mathrm{d}}{\mathrm{d}x} x^n = n x^{n-1}. $$

Proof: The proof is by induction. As a basis for induction, we show that the result holds for $n=1$: $$ \frac{\mathrm{d}}{\mathrm{d}x} x^1 = \lim_{h\to 0} \frac{(x+h)^1 - x^1}{h} = \lim_{h\to 0} \frac{x+h-x}{h} = \lim_{h\to 0} \frac{h}{h} = 1, $$ which is the desired result, and provides a basis for induction.

For induction, suppose that $$ \frac{\mathrm{d}}{\mathrm{d}x} x^k = k x^{k-1} $$ for some $k\in\mathbb{N}$. This assumption is called the induction hypothesis. The goal now is to show that this assumption implies that $$ \frac{\mathrm{d}}{\mathrm{d}x} x^{k+1} = (k+1) x^{k}. $$ This follows from the product rule, shown above: \begin{align} \frac{\mathrm{d}}{\mathrm{d}x} x^{k+1} &= \frac{\mathrm{d}}{\mathrm{d}x} \left[ x \cdot x^{k} \right] \\ &= \left[ \frac{\mathrm{d}}{\mathrm{d}x} x \right] x^{k} + x \left[ \frac{\mathrm{d}}{\mathrm{d}x} x^k \right] \tag{product rule} \\ &= 1 \cdot x^{k} + x \left[ \frac{\mathrm{d}}{\mathrm{d}x} x^k \right] \tag{base case} \\ &= x^k + x \cdot kx^{k-1} \tag{induction hypothesis} \\ &= x^k + k x^k \tag{algebra} \\ &= (k+1) x^k, \tag{more algebra} \end{align} which is the desired result. That is $$ \frac{\mathrm{d}}{\mathrm{d}x} x^k = k x^{k-1} \implies \frac{\mathrm{d}}{\mathrm{d}x} x^{k+1} = (k+1) x^{k}, $$ which completes the induction proof. //

Assuming you want to know $$ \lim_{h\to0}\frac{(x+h)^{123}-x^{123}}{h} $$ set $n=123$. In other words, compute $$ \lim_{h\to0}\frac{(x+h)^{n}-x^{n}}{h} $$ for any integer value of $n$.

Let's show that $(x+h)^n=x^n+nhx^{n-1}+h^2P_n(x,h)$, where $P_n$ is a suitable polynomial in $x$ and $h$.

This is clearly true for $n=1$; so, assume it holds for $n$. Then \begin{align} (x+h)^{n+1} &=(x+h)^n(x+h) \\[6px] &=\bigl(x^n+nhx^{n-1}+h^2P_n(x,h))(x+h) \\[6px] &=x^{n+1}+hx^n+nhx^n+nh^2x^{n-1}+h^2(x+h)P_n(x,h) \\[6px] &=x^{n+1}+(n+1)hx^n+h^2P_{n+1}(x,h) \end{align} where $P_{n+1}=nx^{n-1}+(x+h)P_n(x,h)$ is a polynomial.

The proof by induction is complete.

Then $$ \lim_{h\to0}\frac{(x+h)^n-x^n}{h}= \lim_{h\to0}\bigl(nx^{n-1}+hP_n(x,h)\bigr)=nx^{n-1} $$ For $n=123$, your limit is $123x^{122}$.