$\frac{\partial F}{\partial y}\neq0\implies$ continuous contour line? (Implicit Function Theorem)

Yes, it's correct. Here's a sketch of the argument. Note first of all that by the implicit function theorem, the portion of a level curve of $F$ in the given region must be a one-dimensional manifold with no boundary points in the open rectangle $(x_1,x_2)\times\Bbb R$.

Let $\Gamma$ be the connected component of the level curve $F(x,y)=c$ passing through $(x_1,y_1)$. The fact that $\partial F/\partial y\ne 0$ tells us that $\Gamma$ can have no vertical tangent line, and this says that the set of $x$-coordinates of points on $\Gamma$ cannot have a maximum value $<x_2$. That is, $\Gamma$ contains a point $(x_2,y^*)$. But now the condition $\partial F/\partial y<0$ tells us that level curves of $F$ can have at most one point for each fixed $x$. Therefore, $y^*=y_2$ and $(x_2,y_2)\in\Gamma$.

There may not be a contour line between the two points. Indeed, consider the function $$F(x,y) = -(x^2+1) \arctan y$$ with $\frac{\partial F}{\partial y}(x,y) = -\frac{x^2+1}{y^2+1}< 0$ on $\mathbb{R}^2$. At $(\pm 1, 1)$ we have the value $c=-\frac{\pi}{2}$, but the level curve $$-(x^2+1)\arctan y = -\frac{\pi}{2}$$ does not intersect the axis $x=0$.

The idea is simple: for every $x$ fixed, the function $F(x,y)$ is strictly decreasing in $y$, with image an open interval $I_x$. As $x$ varies, this interval varies too. It may be possible that the interval $I_x$ contains $c$ for $x=x_1$, $x_2$, but, for some intermediate point $x_3$, $I_{x_3}$ does not contain $c$. The example is chosen so that the $c$ will be one end of the interval $I_{x_3}$.

Note: The level curve $F(x,y) = -\frac{\pi}{2}$ is the graph of the function $y = \tan \frac{\pi}{2(x^2+1)}$ defined on $\mathbb{R} \backslash \{0\}$.