Find volume of region bound by $y=x, y=x^2$ around x-axis
Consider the diagram
For your problem, this is equivalent to rotating $R_2$ about the line $y=0$.
Since neither of the curves are touching the axis we are rotating about, we must use washers.
For washers, the volume is given by $$ V = \pi\int^b_a (R(x))^2-(r(x))^2 \ dx$$
Where $R(x)$ is the curve farthest away from our axis of rotation - the top function $y = x$ and $r(x)$ is the curve closest to the axis of rotation - the bottom function $y=x^2.$
As you found the intersecting points, we can set up our integral as
$$V = \pi\int^1_0 (x)^2-(x^2)^2 \ dx$$
$$V = \pi\int^1_0 x^2-x^4 \ dx$$
After integrating and evaluating, you should get $\frac{2\pi}{15}$.
Your formula should be $$V=\pi\int_0^1x^2-(x^2)^2\,dx,$$ which will give you the correct answer. You were trying to use the cylinder method (which would be appropriate if rotating about the $y$-axis), when you should have been using the washer method (since we are integrating along the axis of rotation, rather than perpendicular to it).
In this problem, it is probably easiest to use the method of slicing. However, if you are going to use the "cylindrical shells" method, while rotating about the $x$-axis, you have to integrate with respect to $y$.
Take a thin horizontal strip going from height $y$ to height $y+dy$. The strip is more or less at height $y$. It has length $\sqrt{y}-1$ (draw a picture). The volume of the shell obtained by rotating this strip is about $2\pi y(\sqrt{y}-y)\,dy$. Thus our volume is $$\int_{y=0}^1 2\pi y(\sqrt{y}-y)\,dy.$$ So we want $$2\pi\int_{y=0}^1 (y^{3/2}-y^2)\,dy.$$ Integrate. We get $2\pi\left(\frac{2}{5}-\frac{1}{3}\right)$.