Transforming integral to polar coordinates

You have correctly converted the region of integration and thus we have:

\begin{align} I &= \int_0^{\frac{\pi}{4}}\int_1^{\sec(\theta)} \frac{1}{\left(1 + r^2\cos^2(\theta)\right)\left(1 + r^2\sin^2(\theta)\right)} \cdot r\:dr\:d\theta \\ \end{align}

Applying a partial fraction decomposition we arrive at: \begin{align} I &= \int_0^{\frac{\pi}{4}}\int_1^{\sec(\theta)} \frac{1}{\left(1 + r^2\cos^2(\theta)\right)\left(1 + r^2\sin^2(\theta)\right)} \cdot r\:dr\:d\theta \\ &= \int_0^{\frac{\pi}{4}}\int_1^{\sec(\theta)} \frac{1}{\cos^2(\theta) - \sin^2(\theta)}\left[\frac{\cos^2(\theta)}{1 + r^2\cos^2(\theta)} - \frac{\sin^2(\theta)}{1 + r^2\sin^2(\theta)} \right] r\:dr\:d\theta \\ &= \int_0^{\frac{\pi}{4}}\frac{1}{\cos^2(\theta) - \sin^2(\theta)}\int_1^{\sec(\theta)}\left[\frac{r\cos^2(\theta)}{1 + r^2\cos^2(\theta)} - \frac{r\sin^2(\theta)}{1 + r^2\sin^2(\theta)} \right]\:dr\:d\theta \\ &=\int_0^{\frac{\pi}{4}}\frac{1}{\cos^2(\theta) - \sin^2(\theta)}\cdot\frac{1}{2}\bigg[\ln\left|1 + r^2\cos^2(\theta) \right| + \ln\left|1 + r^2\sin^2(\theta) \right| \bigg]_1^{\sec(\theta)} \:d\theta \end{align}

You okay from here?

Edit: The rest of the solution:

\begin{align} I &= \int_0^{\frac{\pi}{4}}\frac{1}{\cos^2(\theta) - \sin^2(\theta)}\cdot\frac{1}{2}\bigg[\ln\left|1 + r^2\cos^2(\theta) \right| + \ln\left|1 + r^2\sin^2(\theta) \right| \bigg]_1^{\sec(\theta)} \:d\theta \\ &= \int_0^{\frac{\pi}{4}}\frac{1}{\cos^2(\theta) - \sin^2(\theta)}\cdot\frac{1}{2}\bigg[\ln(2) + \ln\left|\sec^2(\theta) \right| \bigg] \:d\theta \\ &= \int_0^{\frac{\pi}{4}}\frac{1}{\cos(2\theta)}\cdot\frac{1}{2}\ln\left|2\sec^2(\theta)\right|\:d\theta\int_0^{\frac{\pi}{4}}\frac{\ln\left| \sqrt{2}\sec(\theta)\right|}{\cos(2\theta)}\:d\theta \end{align}