Find the probability density function of $Y=X^2$

Note that $$F_Y(y)=P(Y\leq y)=P(X^2\leq y)=P(X\leq\sqrt{y})=F_X(\sqrt{y})=\int_0^\sqrt{y}3t^2\,dt=y^{3/2}$$ hence $$f_Y(y)=\frac{3}{2}\sqrt{y}$$

This holds for $0<y<1$.

This theorem will be helpful

$\mathbf {Theorem:}$

Let $X$ have density function $f_X$ and let $A=\{x~:~f_X(x)>0 \}$. If $g$ is strictly monotone on $A$, then $Y=g(X)$ has density

$$f_Y(y)=f_X(g^{-1}(y)) \vert\frac{\partial}{\partial y}g^{-1}(y)\vert.$$

One method is to integrate wrt $x$ to find the CDF and then differentiate it back wrt $y$.

Another is to use the "change of variable transformation", which involves one differentiation.

$\begin{align} \\ f_Y(y) & = f_X{\circ}x(y)\; \Bigg|\frac{\mathrm d x(y)}{\mathrm d y}\Bigg|\tag{1: change of variable} \\[1ex] f_X(x) & = 3x^2\tag{2 given} \\[1ex] y & = x^2\tag{3 given} \\[1ex] x(y) & = \pm\surd y\tag{4 $\impliedby 3$} \\[1ex] \frac{\mathrm d x(y)}{\mathrm d y} & = \mp\tfrac 1 2 y^{-1/2}\tag{$5\impliedby 4$} \\[2ex] f_Y(y) & = f_X{\circ}x(y)\; \Bigg|\frac{\mathrm d x(y)}{\mathrm d y}\Bigg|\tag{1} \\[1ex] & = \frac{3y}{|2\surd y|}\tag{$\impliedby 2,4,5$} \\[1ex] & = \tfrac 3 2 \surd y\tag{7} \\[1ex]\sup(f_X) = [0,1] & \implies \sup(f_Y)=[0,1]\tag{8$\impliedby 3$} \end{align}$

---

This is because: $\displaystyle\frac{\operatorname d}{\operatorname dy}\int_{\sup(f_X)} f_X(x) \operatorname d x = \int_{\sup(f_Y)} f_X{\circ}x(y) \bigg|\frac{\operatorname d x(y)}{\operatorname d y}\bigg|\operatorname d y$

So if we do not need the CDF, this technique might save effort.   It is also handy when the pdf can not be readily integrated (see: Gaussian Normal Distributions).