Why the photon can't produce electron and positron in space or in vacuum?

You can't simultaneously conserve energy and linear momentum.

Let the photon have energy $E_{\gamma} = p_{\gamma} c$ and the electron have energy $E_{-}^{2} = p_{e}^{2}c^2 + m_{e}^{2}c^4$ and an analogous expression for the positron. Suppose the electron and positron depart from the interaction site with an angle $2\theta$ between them.

Conservation of energy.

$$ p_{\gamma} c = \sqrt{(p_{e}^{2}c^2 +m_e^{2}c^4} + \sqrt{(p_{p}^{2}c^2 +m_e^{2}c^4},$$ but we know that $p_{p} = p_{e}$ from conservation of momentum perpendicular to the original photon direction. So $$ p_{\gamma} = 2\sqrt{p_{e}^2 + m_e^{2}c^2}$$

Now conserving linear momentum in the original direction of the photon. $$p_{\gamma} = p_e \cos{\theta} + p_p \cos\theta = 2p_e \cos\theta$$

Equating these two expression for the photon momentum we have $$p_e \cos{\theta} = \sqrt{p_{e}^2 + m_e^{2}c^2}$$ $$\cos \theta = \sqrt{1 + m_e^{2}c^2/p_e^{2}}$$ As $\cos \theta$ cannot exceed 1 we see that this impossible.

Let us say that the photon has created pair of massive particles. There must exist a reference frame ("center of mass") in which the total momentum of the two particles is zero. So by momentum conservation, in the same frame the photon had to have momentum zero. But photon cannot have momentum zero, because then it would have zero energy. So momentum is not conserved in the center of mass frame, and, because of the Lorentz invariance, is not conserved in all frames. Therefore, the process is impossible.