Relation between velocity and position vector in a central force

Edit: As requested, here is a calculation for the ellipsis. An ellipsis can be parametrically described by $\vec r(t) = (a\cos \varphi(t),\, b\sin\varphi(t))^\top$ where $0\leq\varphi(t) < 2\pi$. Then, $$\vec v(t) = \dot{\vec{r}}(t) = (-a\dot{\varphi}(t)\sin\varphi(t),\, b\dot{\varphi}(t)\cos\varphi(t))^\top.$$ Now take the dot product $$ \vec{r}(t)\cdot\vec{v}(t) = -a^2\dot\varphi(t)\sin\varphi(t)\cos\varphi(t) + b^2\dot\varphi(t)\sin\varphi(t)\cos\varphi(t). $$ We can see, that it vanishes for all times $t$ if and only if $a=b$ (i.e. the orbit is circular).

The position and velocity are not always perpendicular to each other for an elliptical orbit. This can be easily seen if you draw an ellipsis and add the position and velocity vectors.

Actually, the position is perpendicular to the velocity if and only if the orbit is a circle. This is because only then the radius remains constant, which is in fact the definition of a circle. If it is not the radius must change over time. In order to formally proof it, consider $$ \frac{d}{dt}\lVert\vec r(t)\rVert^2 = \frac{d}{dt}[\vec r(t)\cdot\vec r(t)] = 2\vec r(t)\cdot \dot{\vec r}(t) = 0, $$ since $\vec r\perp\vec v$. Hence, the distance is constant, meaning that we have a circular orbit. For the other direction see the above calculation for the ellipsis. $$\tag*{$\Box$}$$

The deduction of the expression for $\vec{r} \cdot \dot{\vec{r}}$, where $\vec{r}$ is the displacement of the center of mass of the orbiting entity from the center of mass of the entity which it orbits and $\vec{v} := \dot{\vec{r}}$ is the velocity of the orbiting entity, is critical to the vector analysis which usually accompanies an orbital mechanics study. Denoting $r:=|\vec{r}|$ and $\hat{r} := \frac{\vec{r}}{|\vec{r}|}$, we can see that $\dot{|\hat{r}|} := \frac{d |\hat{r}|}{dt} = \dot{1} = 0 = \vec{r} \cdot \dot{\vec{r}}$, which implies that $\dot{\hat{r}} \perp \hat{r}$. Indeed, in general, the time derivative of a vector of constant magnitude is perpendicular to the vector. Therefore, $\vec{r} \cdot \dot{\vec{r}} = r \hat{r} \cdot (\dot{r} \hat{r} + r \dot{\hat{r}})$, so that $\vec{r} \cdot \vec{v} = r \dot{r}$.

Clearly, $\vec{r} \cdot \vec{v} = 0$ if and only if the motion is circular, that is $\dot{r} \equiv 0$. On the other hand, if the motion is not circular, the inner product of interest does not, in general, vanish. Notice that we did not have to use any property of the force field to arrive at this result, since the question asked by the OP is in fact a question on kinematics and does not involve the dynamics.