Characterization of the matrix inverse

The answer is No, here is a counter example $$A=\left[\matrix{0&1\cr 1&0}\right],\quad X=\left[\matrix{1&1\cr 1&-1}\right]$$ here $A^{-1}=A\ne X$ while $AX+XA=2I$.