Graphs with many edges avoided by Hamiltonian cycles
The computer found counterexamples to $\rho(G)<1$.
Despite verification, I am not sure this is correct.
$G_1$ on $7$ vertices, $G_2$ on $11$ vertices. $\rho(G_1)=1,\rho(G_2)=2$
$G_1$:
edges=[(0, 3), (0, 4), (0, 5), (1, 4), (1, 5), (1, 6), (2, 4), (2, 5), (2, 6), (3, 5), (3, 6), (4, 6)]
A=[(0, 3)]
B=[(4, 6)]
$G_2$:
edges=[(0, 5), (0, 6), (0, 7), (1, 6), (1, 7), (1, 8), (2, 6), (2, 8), (2, 9), (3, 7), (3, 9), (3, 10), (4, 8), (4, 9), (4, 10), (5, 9), (5, 10), (6, 8), (7, 10)]
A=[(0, 5)]
B=[(6, 8), (7, 10)]
Plot of $G_1$:
The first counterexample given by @joro gives rise to the following family $G_k$ of graphs with $\rho(G_k)=k$ : $G_k$ has $n=4k+3$ vertices numbered $0,1,\dots,n-1$ forming a H-cycle (imagine a regular n-gon embedded such that the vertex $2k+1$ is on the bottom), plus the ‘horizontal’ chords $(2j-1,4k+3-2j)$ for $j=1,...,k$ (call their set $B$) and the other chords $(j,2k+1+j)$ for $j=0,...,2k+1$. It is not hard to see that $(4k+2,0)$ is an a-edge. As the graph without this edge and without $B$ is bipartite, all edges of $B$ must be b-edges. It remains to check that for each other edge, there is a H-cycle that doesn’t contain it.
For the second question, as @Martin Tancer and @nvcleemp have pointed aout, $\rho(G)\le1/2$ for cubic graphs. The following construction comes arbitrarily close to $1/2$ :
For $n=2k$, start with a H-cycle $0,...,2k-1,0$ and add the ‘horizontal’ chords $(j,2k-j)$ for $j=1,...,k-1$ and the ‘vertical’ chord $(0,k)$. Replace the vertex $k$ with an instance of $P$ that blocks the vertical chord. Then it is easy to see (starting at the vertex $0$) that the horizontal chords cannot be part of a H-cycle. So this graph has $a(G)=2k+7$ (the original cycle plus 7 inner edges of $P$) and $b(G)=k+1$ (the horizontal chords plus 2 inner edges of $P$).
So both original questions are answered, thanks to your input. Remains the interesting question raised by joro what can be said about $\rho(G)$ for 4-regular graphs.