How to prove that factors of homogeneous polynomial are homogeneous?

This problem is easier if we use another formulation of $f$ being a homogeneous polynomial.

First I'll prove the following fact: a polynomial is homogeneous (in your definition) if and only if each monomial appearing in $f$ has total degree $n$.

Proof: First suppose that $f$ is homogeneous. Write $f$ as a sum of monomials $f_i$ of total degree $i$: $f=f_0+f_1 + \ldots+f_r$. Then $$ a^nf(x) = f(ax)=f_0(ax)+f_1(ax)+\ldots+f_r(ax)=f_0+af_1(x)+\ldots+a^rf_r(x). $$ We want to prove that $n=r$ and that $f_r$ is the only term in $f$. For the first fact, note that both sides of the equations are polynomials in $a$. The left hand side has degree $n$ and the right hand side has degree $r$. Hence $n=r$.

If we put $a=0$, we see that $f_0=0$. Now differentiate with respect to $a$ to get $$ na^{n-1}f(x) = f_1(x)+2af_2(x) + \ldots + ra^{r-1}f_r(x). $$ Put $a=0$. Then we see that $f_1=0$. Continue, to conclude that $f(x)=f_r(x)$. Hence we have shown that a homogeneous polynomial must consist of a sum of monomials of total degree $n$.

The other direction is easy and is left to the reader.

--

Now for your question. We want to see that the factors of a homogeneous polynomial are themselves homogeneous. We do this for two factors, since the general case follows by induction.

So suppose that $f=gh$, where $f$ is homogeneous, but one of $g,h$ is not. So suppose $g$ is not homogeneous. By the above, this means that we can write $g=g_{ih}+g_h$, where $g_{h}$ is $g$'s highest degree term, and $g_{ih}$ is its "inhomogeneous component" (=the rest). We have $$ f=gh=(g_{ih}+g_h)h=g_{ih}h+g_hh. $$ For degree reasons (the first term on the right hand side has degree less than $f$, which consist of only top degree monomials), we must have $g_{gih}h=0$, but $g_{ih}$ was nonzero, so $h=0$, but this is absurd.

Let $\widehat{f}(x_1,x_2,\dots,x_n,t)=f(tx_1,\dots,tx_n)$ in the polynomial ring $k[x_1,\dots,x_n,t]$. Then $f$ is homogeneous if and only if $$ \widehat{f}(x_1,x_2,\dots,x_n,t)=t^nf(x_1,\dots,x_n) $$ This is different from your definition which is equivalent to this one only over infinite fields. For instance, with your definition, every polynomial over the two element field would be homogeneous.

Suppose $f=gh$; then we have $$ \widehat{f}(x_1,\dots,x_n,t)= f(tx_1,\dots,tx_n)= g(tx_1,\dots,tx_n)h(tx_1,\dots,tx_n) $$ Now we can write \begin{align} g(tx_1,\dots,tx_n)&=g_0+g_1t+\dots+g_at^a,\\ h(tx_1,\dots,tx_n)&=h_0+h_1t+\dots+h_bt^b, \end{align} with $g_i,h_j\in k[x_1,\dots,x_n]$, $g_a\ne0$ and $h_b\ne0$. Suppose $f$ is homogeneous; consider the least integer $c$ such that $g_i=0$ for $i<c$, $g_c\ne0$ and the least integer $d$ such that $h_j=0$ for $j<c$ and $h_d\ne0$.

Note that $a+b=n$, $c\le a$ and $d\le b$.

Then the term of degree $c+d$ in $g(tx_1,\dots,tx_n)h(tx_1,\dots,tx_n)$ is (with $g_i=0$ for $i>a$ and $h_j=0$ for $j>b$ and consider everything in the ring of polynomials in $t$ with coefficients in $k[x_1,\dots,x_n]$) $$ g_0h_{c+d}+g_1h_{c+d-1}+\dots+ g_{c-1}h_{d+1}+g_ch_d+g_{c+1}h_{d-1}+\dots+g_{c+d}h_0 =g_ch_d\ne0 $$ Since $f$ is homogeneous, $\widehat{f}$ has only the leading coefficient (at degree $n$) non zero, as a polynomial in $t$ with coefficients in $k[x_1,\dots,x_n]$, so $c+d=n$ and therefore $c=a$, $d=b$.

Therefore $$ \widehat{g}(x_1,\dots,x_n,t)=t^ag_a(x_1,\dots,x_n) $$ Evaluating at $t=1$, $$ g(x_1,\dots,x_n)=\widehat{g}(x_1,\dots,x_n,1)=g_a(x_1,\dots,x_n) $$ so we have proved that $$ \widehat{g}(x_1,\dots,x_n,t)=t^ag(x_1,\dots,x_n) $$ and therefore $g$ is homogeneous. Similarly for $h$.