Prove $n^2(\log n)^{10} = O(n^{2,1})$

You do need to show that a suitable constant $c$ exists, together with a constant $K$ such that the inequality holds for $n\ge K$. But you don't need to compute the numbers explicitly -- it's enough if you can show an arithmetic expression for them, for example.

There are many combinations of $c$ and $K$ that work, but a smaller $K$ generally requires a larger $c$ and vice versa. So there is not a unique "best" pair to find. This means that you shouldn't hope to derive concrete values for $c$ and $K$ out of thin air; somewhere you need to make a fairly arbitrary choice.

You have a good start, but the calculations for find $c,K$ that gives $(\log n)^{10} \le cn^{1/10}$ will be simpler if you now switch the variable from $n$ to $m=\frac{\ln n}{10}$. That gets rid of both the fractional exponent and the logarithm. (Convince yourself that compressing the $n$-axis of the functions does not change the asymptotic relations).


A small caveat: Once you're past the stage where you're proving things like $n^2\log^{10} n = O(n^{2.1})$ as exercises in their own right, you will be able to (and expected) to get away with just stating such things as fact because "everybody knows" larger exponents always win over any number of logarithmic factors.


Often it is useful to know the constant. It can be estimated as follows.

Start with $$ \log(n)\lt\log(1+n)=\int_0^n\frac{\mathrm{d}x}{1+x}\le n $$ Thus, $$ \frac1{100}\log(n)\le n^{1/100} $$ Raise both sides to the $10^\text{th}$ power: $$ \log(n)^{10}\le10^{20}n^{1/10} $$ Therefore, $$ n^2\log(n)^{10}\le10^{20}n^{21/10} $$