Functions that satisfy $kf(x) \leq f(x/k)$ for $k \in (0,1]$

There are many such functions. Let $g$ be any positive increasing continuous function on $\mathbb R^{+}$. Then $f(x)=g(|x|)$ satisfies this inequality since $kf(x)\leq f(x)=g(|x|) \leq g(|\frac x k|)=f(\frac x k)$.

One special case: $g(x)=x^{\alpha}$ with $\alpha >0$. So any linear combination of the functions $|x|^{\alpha}$ with non-negative coefficients is also a solution.

One necessary condition is $f \geq 0$ even if we know the inequality for one fixed $k<1$ : $f(x) \geq kf(kx)$ so $f(x) \geq k^{n}f(k^{n}x) \to (0)(f(0))=0$.

Another necessary condition is $xf(x)$ must be increasing on $(0,\infty)$. To prove this let $0<x<y$. Then $x=ky$ where $k=\frac x y \in (0,1)$. Hence $kf(x)=kf(ky)\leq f(y)$ which means $\frac x y f(x) \leq f(y)$ or $xf(x) \leq yf(y)$. This condition is also sufficient on $(0,\infty)$: if $xf(x)$ is increasing on $(0, \infty)$ the the given inequality holds for $x>0$ and $0<k \leq 1$