$2x^2 + 3x +4$is not divisible by $5$

When you want to get rid of an $x$ term in a quadratic, you should always try completing the square! We find that $$2x^2+3x+4=2((x+2)^2-2)$$ If this is $0\pmod{5}$, then so is $(x+2)^2-2$. But you've already pointed out that $\not\exists k\in\mathbb Z$ such that $k^2=2\pmod{5}$ so that's impossible.

The quadratic formula works in all fields whose characteristic is not $2$. Since you are working in $\mathbb{Z}_5$ we are safe.

The quadratic formula gives that the solutions would be:

$$ x = \frac{-3 \pm \sqrt{3^2 - 4(2)(4)}}{8} = \frac{-3 \pm \sqrt{-23}}{8} = \frac{-3 \pm \sqrt{2}}{8}_.$$

You can manually check that $\sqrt{2}$ does not exist in $\mathbb{Z}_5$. So no roots exist in $\mathbb{Z}_5$.

$5$ is small. You could just consider all possibilities $\mod{5}$. There are only $0,1,2,3,4$. If any of these get you $0$, then, it can be possibly divisible by $5$. Otherwise, not. Let $f(x)=2x^2+3x+4$. So, $x\equiv0\implies f(0)\equiv4$, $x\equiv1\implies f(1)\equiv 2+3+4\equiv4\pmod{5}$, $x\equiv2\implies f(2)\equiv8+6+4\equiv3$, $x\equiv3\implies f(3)\equiv 18+9+4\equiv 1\pmod{5}$, and finally $x\equiv4\implies f(4)\equiv 32+12+4\equiv3$. An even faster way to do the calculations would just be to see that $3\equiv-2$ and $4\equiv-1$.