How to show that $f(x) = x^2$ is continuous using topological definition?
Open intervals $(a,b), a < b$ form a base for the topology of $\mathbb{R}$.
What is $f^{-1}[(a,b)] = \left\{x \in \mathbb{R}: x^2 \in (a,b) \right\}$?
If $b \le 0$, then no square is in $(a,b)$ so then $f^{-1}[(a,b)] = \emptyset$, which is open. So assume $b > 0$. Then $x^2 < b$ iff $x \in (-\sqrt{b},\sqrt{b})$. If $a < 0$, the $a$ does not impose an extra condition, as all $x^2 \ge 0 > a$ in that case, so
$f^{-1}[(a,b)] = (-\sqrt{b},\sqrt{b})$ if $b > 0, a < 0$, which is an open interval in $\mathbb{R}$ so open.
Otherwise we also need $x^2 > a \ge 0$, so $x < -\sqrt{a}$ or $x > \sqrt{a}$ respectively.
So then $f^{-1}[(a,b)] = (-\sqrt{b}, -\sqrt{a}) \cup (\sqrt{a},\sqrt{b})$ if $b > a \ge 0$, which is open as the union of two open intervals.
This covers all cases, so $f^{-1}[(a,b)]$ is open for all intervals.
Now if $O$ is open, we can write $O = \bigcup_{i \in I} (a_i,b_i)$, for some family of open intervals $(a_i,b_i), i \in I$, as the intervals form a base. But then
$$f^{-1}[O] = f^{-1}[\bigcup_{i \in I} (a_i,b_i)] = \bigcup_{i \in I} f^{-1}[(a_i,b_i)]$$
by standard properties of $f^{-1}$ and the last set is open as unions of open sets are open, and we have shown that the inverse images of the base sets are open.