How to solve an definite integral of floor valute function?

Related problems: (I), (II). Using the change of variables $ u=\sqrt{t} $ makes the problem to fall apart $$ \int_0^{n^2}\lfloor\sqrt{t}\rfloor dt = 2\int_0^{n}\lfloor u \rfloor u du = 2\sum_{k=0}^{n-1}k\int_{k}^{k+1} u \, du = \sum_{k=0}^{n-1} k\,( (k+1)^2 - k^2 ) $$

$$ = \frac{1}{6}n(n-1)(4n+1). $$

Note 1: To evaluate the sum, you may need to look here.

Clarification: Answering your comment. In the above calculations, since $ \lfloor u \rfloor = k,\quad k\leq u < k+1 $, then

$$ \int_0^{n} \lfloor u \rfloor u du = \int_0^{1} (0) u du+ \int_1^{2} (1) u du + \int_2^{3} (2) u du + \dots + \int_{n-1}^{n} (n-1) u du $$

$$ = \sum_{k=0}^{n-1}\int_{k}^{k+1} (k)\, u \, du. $$

Consider the definite integral as the area under the curve, noting that $n$ is an integer. You may find this sketch helpful.

Perhaps a nicer way of wording the problem would be:
Show that $$\int_{0}^{n^{2}}\lfloor \sqrt{t}\rfloor dt=\sum_{r=0}^{n-1}r(2r+1)$$

More detailed: You want to find the area under the curve $y=\lfloor \sqrt{x} \rfloor$ as $x$ goes from $0$ to $n^{2}$, where $n$ is an integer.

Well, consider it over the range $0\le x<1$. Then $0\le \sqrt{x}<1$, so $\lfloor \sqrt{x} \rfloor=0$. Hence, the area over the interval $0\le x<1$ is the length of the interval times its height. So, $1\cdot 0=0$.

Now consider the next range, $1\le x<4$, so that $1\le \lfloor \sqrt{x} \rfloor < 2$. The height in this interval is $1$, by the same logic as before, and its width is $3$. Hence, the area between $1$ and $4$ is $3$.

Now think more generally: in the $n^{th}$ case, $(n-1)^2 \le x < n^{2}$, so that $(n-1) \le \lfloor \sqrt{x} \rfloor < n$. The height is $n-1$ and the width is $2n-1$. You should be able to now show that the integral is equal to the sum on the RHS above. @Cocopuffs' answer shows how to compute the sum.

$$\int_0^{n^2} \lfloor \sqrt{t} \rfloor dt = \sum_{k=0}^{n^2-1} \int_k^{k+1} \lfloor \sqrt{t} \rfloor dt$$ $$=\sum_{k=0}^{n^2 - 1} \lfloor \sqrt{k} \rfloor$$ $$=\sum_{j=0}^{n-1} j\Big((j+1)^2 - j^2\Big)$$ $$= 2\sum_{j=0}^{n-1} j^2 + \sum_{j=0}^{n-1} j$$ $$= 2\frac{(n-1)n(2n-1)}{6} + \frac{3(n-1)n}{6}$$ $$=\frac{(n-1)n(4n+1)}{6}.$$