What is the greatest possible radius of a circle that passes through the points (1, 2) and (4, 5), whose interior is contained Q1?

In the first quadrant, either axis would limit the size of the circle. Since the points $(1,2)$ and $(4,5)$ are further away from the $x$-axis than from the $y$-axis, the circle with the largest area is expected to touch the $x$-axis.

So, the corresponding equation for the circle with center $(a,b)$ takes the form,

$$(x-a)^2+(y-b)^2=b^2$$

where its radius is just $b$, the y-coordinate of the center. Plug the two points $(1,2)$ and $(4,5)$ into above equation,

$$(1-a)^2-4b+4=0$$ $$(4-a)^2-10b+25=0$$

The solution for $b$ is

$$b=7-2\sqrt{5}$$

which is also the radius of the largest circle.

Consider the $2$ points to be $A(1,2)$ and $B(4,5)$. The center of any circle passing through these $2$ points must be perpendicular bisector of $AB$. The slope of $AB$ is $\frac{5-2}{4-1} = 1$, so the slope of the perpendicular bisector is the negative reciprocal, i.e., $-1$. Also, the midpoint of $AB$ is $M(\frac{1+4}{2},\frac{5+2}{2}) = M(\frac{5}{2},\frac{7}{2})$. Thus, if the perpendicular bisector line's formula is of the form $y = mx + b$, with $m = -1$, you get $\frac{7}{2} = -\frac{5}{2} + b \implies b = 6$. Thus, the perpendicular bisector line's formula is

$$y = -x + 6 \tag{1}\label{eq1}$$

Consider a point $C(t, -t + 6)$ along the line in \eqref{eq1} to be the center point of a circle through $AB$. Let $r$ be the radius of this circle. For the entire circle to be in the first quadrant requires that

$$r \le t \implies r^2 \le t^2 \tag{2}\label{eq2}$$

and

$$r \le -t + 6 \implies r^2 \le (-t + 6)^2 = t^2 - 12t + 36 \tag{3}\label{eq3}$$

Next, note the lengths of $AC$ and $BC$ are equal to each other and to $r$. Consider just $AC$. This give the equation, when the distance is squared, of

$$\begin{equation}\begin{aligned} r^2 & = (t - 1)^2 + (-t + 6 - 2)^2 \\ r^2 & = (t - 1)^2 + (t - 4)^2 \\ r^2 & = t^2 - 2t + 1 + t^2 - 8t + 16 \\ r^2 & = 2t^2 - 10t + 17 \end{aligned}\end{equation}\tag{4}\label{eq4}$$

From \eqref{eq2}, this gives

$$2t^2 - 10t + 17 \le t^2 \implies t^2 - 10t + 17 \le 0 \tag{5}\label{eq5}$$

and, from \eqref{eq3}, \eqref{eq4} gives

$$2t^2 - 10t + 17 \le t^2 - 12t + 36 \implies t^2 + 2t - 19 \le 0 \tag{6}\label{eq6}$$

The maximum radius occurs where \eqref{eq5} or \eqref{eq6} is $0$. With \eqref{eq5}, the roots are $t = 5 \pm 2\sqrt{2}$. With $r = t = 5 - 2\sqrt{2}$, you have $-t + 6 \gt r$, so eqref{eq3} doesn't hold. Since $t = 5 + 2\sqrt{2} \gt 6$ means $-t + 6 \le 0$, so it's not a valid root. With \eqref{eq6}, the roots are $t = -1 \pm 2\sqrt{5}$. Since $t \gt 0$, the only valid root is $t = -1 + 2\sqrt{5}$, so $r = -t + 6 = 7 - 2\sqrt{5}$ (and \eqref{eq2} also holds), with this being the maximum radius.