Conditions on matrices imply that 3 divides $n$

Assembling the comments, I write the entire solution.

STEP 1: the first hypothesis in characteristics 3 is equivalent to $(X-Y)^2 = [X,Y]$.

Indeed, note that

$$(X-Y)^2 - [X,Y] = X^2-XY-YX+Y^2- XY+YX = X^2-2XY+Y^2 = X^2+Y^2+XY$$

Why did I make this calculation? Note that $X^2+Y^2+XY$ resembles a factor of $X^3-Y^3$, which would be equal (in commutative case) to $(X-Y)^3$ in char 3. To get there we would need a $X-Y$ more, so we can guess that $(X-Y)^2$ and $X^2+Y^2+XY$ will be equal up to some commutators!

STEP 2: getting the final result.

We have that $(X-Y)^2 = [X,Y]$ is invertible by the second hypothesis. Thus in particular $X-Y$ is invertible. This allows us to substitute $X=M+Y$ with M invertible, obtaining

$$M^2 = [M+Y,Y] = [M,Y]$$

The only relevant information we have about arbitrary commutators is that the trace is zero. We hence would like to have some matrix with easy trace (like the identity) to compare witha commutator. To do this, let's multiplicate on both the left and the right for $M^{-1}$:

$$ I = M^{-1}[M,Y]M^{-1} = M^{-1}(MY-YM)M^{-1} = YM^{-1}-M^{-1}Y = [Y,M^{-1}]$$ Taking traces we get $n=0$. Being in characteristics 3, this gives $3 \mid n$.

Let me start from $M^2=[M,Y]$ with $M\in {\bf GL}_n(k)$, as suggested by Andrea. Wlog, I assume that the characteristic polynomial of $M$ splits over $k$, and I decompose $k^n$ as the direct sum of characteristic subspaces $E_\mu=\ker(M-\mu I_n)^n$. It is enough to prove that the dimension of each $E_\mu$ is a multiple of $3$. To this end, observe that $Y$ acts over $E_\mu$. In details, let $x$ be an eigenvector, $Mx=\mu x$. Then $(M-\mu)Yx=\mu^2x$. Because $M$ is invertible, $\mu\ne0$ and therefore $$Yx\in\ker(M-\mu)^2\setminus\ker(M-\mu).$$ Likewise $$(M-\mu)Y^2x=2\mu^2Yx+2\mu^3x,$$ hence $Y^2x\in\ker(M-\mu)^3\setminus\ker(M-\mu)^2$. Eventually, using again ${\rm char}(k)=3$, we have $$(M-\mu)Y^3x=0.$$ We deduce that a basis $E_\mu$ is obtained by taking a basis $\cal B$ of $\ker(M-\mu)$, and adjoining the vectors of $Y\cal B$ and $Y^2\cal B$ ; all of them are linearly independent, as seen above. Thus $\dim E_\mu=3\dim\ker(M-\mu)$.