PDF of $Y - (X - 1)^2$ for $(X, Y)$ uniform on $[0, 2] \times [0, 1]$

Although your final formula is correct (kudos for that), let me advocate a more systematic approach. Be warned though that this approach avoids nearly all head-scratching, hence it is not suitable to anybody preferring this kind of experience (that is, head-scratching) to the banal application of some routine and very sure procedure.

First step: Write down the density correctly.

That is, recall that the density $f$ of the distribution of $(X,Y)$ is defined on the whole space $\mathbb R^2$ hence one should define the function $f$ everywhere and avoid cases. This is easily done, using indicator functions. Here, for every $(x,y)$ in $\mathbb R^2$, $$ f(x,y)=\frac12\mathbf 1_{0\leqslant x\leqslant 2,0\leqslant y\leqslant 1}. $$

Second step: Use the functional approach.

That is, try to reach the identity $$ E[u(Z)]=\int_\mathbb R u(z)g(z)\mathrm dz, $$ for every bounded measurable function $u$. If this is done, one knows that $g$ is the density of $Z$.

To do so, note that, by definition of the distribution of $(X,Y)$ as the measure with density $f$, $$ E[u(\color{red}{Y-(X-1)^2})]=\iint_{\mathbb R^2} u(\color{red}{y-(x-1)^2})f(x,y)\mathrm dx\mathrm dy, $$ hence the goal is to equate the RHS of the two last displayed equations. Note that nothing we wrote until now is case-specific hence these steps will always be the same (boring, I told you).

Third step: Choose a change of variable.

Here, $(x,y)\to(z,t)$ where, obviously, $z=\color{red}{y-(x-1)^2}$ and $t$ is almost free. A plausible choice is $t=x-1$ (but others are equally handy). This is the first moment when one should be half-awake.

One must also express the old variables $(x,y)$ in terms of the new variables $(z,t)$ (second half-awake moment). Here, $(x,y)=(t+1,z+t^2)$.

Fourth step: Proceed with the change of variable by computing the associated Jacobian.

Here, $\mathrm dx=\mathrm dt$ hence $\mathrm dx\mathrm dy=\mathrm dt\mathrm dz$, hence the Jacobian (third half-awake moment) is $1$ and $$ E[u(Z)]=\iint_{\mathbb R^2} u(z)f(t+1,z+t^2)\,1\,\mathrm dz\mathrm dt, $$ which indicates that $$ g(z)=\int_\mathbb Rf(t+1,z+t^2)\mathrm dt, $$ that is, in the present case, $$ g(z)=\int\frac12\mathbf 1_{0\leqslant t+1\leqslant 2,0\leqslant z+t^2\leqslant 1}\mathrm dt=\frac12\int_{-1}^1\mathbf 1_{-z\leqslant t^2\leqslant 1-z}\mathrm dt. $$ Now the identification of $g$, which in general is pretty fast but in your case is actually rather tedious. One has:

• If $z\gt1$, then $t^2\leqslant 1-z$ never happens.
• If $0\lt z\lt1$, then $-z\leqslant t^2\leqslant 1-z$ and $-1\lt t\lt1$ happens when $-\sqrt{1-z}\leqslant t\leqslant\sqrt{1-z}$.
• If $-1\lt z\lt0$, then $-z\leqslant t^2\leqslant 1-z$ and $-1\lt t\lt1$ happens when $-1\leqslant t\leqslant-\sqrt{-z}$ or $\sqrt{-z}\leqslant t\leqslant1$.
• If $z\lt-1$, then $-z\leqslant t^2$ and $-1\lt t\lt1$ never happens.

This shows that $$ \color{green}{g(z)=\sqrt{1-z}\,\mathbf 1_{0\lt z\lt1}+(1-\sqrt{-z})\,\mathbf 1_{-1\lt z\lt0}}. $$

Final step: Check that the result is plausible in every way you can think of.

Do not omit this final step. Here, check at least that $g\geqslant0$ everywhere and that the integral of $g$ on $\mathbb R$ is $1$ (otherwise something went amiss). Et voilà!