Moment of inertia of a sphere

It did not came clear from the other answers why your approach is wrong. Remember that you calculate the moment of inertia for rotation around an axis not around a point. So if you choose for example the vertical axis, you notice that the points on spherical shell are at a constant distance from the center of the sphere, but they are at different distances from the vertical axis. You can use either point like masses, or cylinders along the vertical direction. You need to compute $$\frac{m}{\frac{4}{3}\pi r^3}\iiint_S(x^2+y^2)dxdydz$$ What you did was instead computing the integral of $x^2+y^2+z^2$.

There is a good trick with the sphere, because it is symmetrical. By definition, $$ I_x + I_y + I_z = \sum_i m_i(2x_i^2+2y_i^2+2z_i^2) = \sum_i 2m_i r_i^2 $$ where the sum is over all the point masses we imagine splitting the object into; $(x_i^2 + y_i^2)$ comes from $I_z$, $(x_i^2 + z_i^2)$ comes from $I_Y$ and $(y_i^2 + z_i^2)$ comes from $I_x$ - when you add them up you get the summand above. Now the $r_i$ in that formula is exactly the $r$ you were using when integrating - the distance from the center (and your mistake was using that instead of, say, $x^2+y^2$).

So it turns out that your integral is useful, after all; it tells us that (you didn't have a factor of $2$ in your integral, that's why I include that now) $$ I_x + I_y + I_z = 2\cdot \frac{3}{5}mR^2 $$ But since all the $I$'s are the same by symmetry, $$ I_x = I_y = I_z = \frac{1}{3}\cdot\frac{6}{5}mR^2 = \frac{2}{5}mR^2. $$

Similarly, for a hollow spherical shell, $r_i = R$ for all points, so there's no need to integrate and $$ I_x = I_y = I_z = \frac{1}{3}\cdot 2mR^2. $$