Evaluating $\int_0^{2\pi}\frac{dt}{\sqrt[4]{P(\cos t,\sin t)}}$
This post is a work in progress and I still have a lot of work to do in terms of formatting, showing work and the final evaluation of the integral. I have used contour integration to reduce the integral to an integral involving only polynomials. If you have any questions feel free to ask.
Let $\gamma$ be the unit$\require{autoload-all}$ circle.
$$I = \int_0^{2\pi}\!\!\frac{\mathrm{d}t}{\sqrt[4]{A(\sin^8(t)+\cos^8(t))+B(\sin^6(t)\cos^2(t)+\sin^2(t)\cos^6(t))+C\sin^4(t)\cos^4(t)}}$$
$$ \toggle{ \text{Set} \; x = e^{it}\quad\enclose{roundedbox}{\text{ Click for Information }} }{ \begin{align} x &= e^{it}\\ \sin(t) &= \frac{1}{2i}\left(x-\frac{1}{x}\right)\\ \cos(t) &=\frac{1}{2}\left(x+\frac{1}{x}\right)\\ \mathrm{d}t &= \frac{-i \, \mathrm{d}x}{x} \end{align} }\endtoggle $$
$$I = \int_\gamma\!\frac{-i\,\mathrm{d}x}{x\sqrt[4]{A((x-\frac{1}{x})^8+(x+\frac{1}{x})^8)-B((x-\frac{1}{x})^6(x+\frac{1}{x})^2+(x-\frac{1}{x})^2(x+\frac{1}{x})^6)+C(x^2-\frac{1}{x^2})^4}}$$
We will let $$P'(x) = A\left(\left(\!x-\frac{1}{x}\!\right)^8\!\!+\left(\!x+\frac{1}{x}\!\right)^8\right) \!-B\left(\!x^2\!-\!\frac{1}{x^2}\!\right)^2\!\left(\left(\!x-\frac{1}{x}\!\right)^4\!\!+\left(\!x+\frac{1}{x}\!\right)^4\right)\! + C\left(\!x^2\!-\!\frac{1}{x^2}\!\right)^4$$
and $$P(x) = 256\cdot P'(x)$$
$$I = \int_\gamma \! \frac{-4i}{x\sqrt[4]{P(x)}} \mathrm{d}x$$
We will now proceed to evaluate the integral using contour methods. A careful note suggests that $P(x)$ can be solved like a quartic in $x^4$. Let the roots of this polynomial be $\xi_j$ for $1\leq j \leq 16$. Note that the function we are integrating tends to $0$ as $x\to 0$.
For any given sets of parameters, find the roots and use the theory of contour integration to evaluate the integral. This could be possible in the general case but most likely not worth pursuing.
With this we should be able to find a somewhat closed form for your specific case.
For your parameters, $$\begin{align}P(x) =\frac{86 x^8}{5}+\frac{86}{5 x^8}+\frac{16 x^4}{5}+\frac{16}{5 x^4}+36 \end{align}$$
Because it shares the same roots as a quartic in $x^4$ it admits roots expressible in terms of radicals. $$ \toggle{\enclose{roundedbox}{\text{ Click for Analysis of Roots }} }{ \text{For simplicity, define}\\ \Omega_1 = \Re\left(-\frac{2}{43}-\frac{i \sqrt{39}}{43}+\frac{1}{2} \sqrt{-\frac{7536}{1849}+\frac{16 i \sqrt{39}}{1849}}\right) \approx -0.0398198141\\ \Omega_2 =\Im\left(-\frac{2}{43}-\frac{i \sqrt{39}}{43}+\frac{1}{2} \sqrt{-\frac{7536}{1849}+\frac{16 i \sqrt{39}}{1849}}\right) \approx 0.8642098747 \\ \Omega_3 = \Re\left(-\frac{2}{43}+\frac{i \sqrt{39}}{43}+\frac{1}{2} \sqrt{-\frac{7536}{1849}-\frac{16 i \sqrt{39}}{1849}}\right) \approx -0.0532034417\\ \Omega_4 = \Im\left(-\frac{2}{43}+\frac{i \sqrt{39}}{43}+\frac{1}{2} \sqrt{-\frac{7536}{1849}-\frac{16 i \sqrt{39}}{1849}}\right) \approx 1.1546748979 \\ \text{The roots are given by}\\ \begin{align} \zeta_{1-4} &= (\Omega_1+i\Omega_2 )^{1/4}e^{k\pi i/2}\\ \zeta_{5-8} &= (\Omega_1-i\Omega_2 )^{1/4}e^{k\pi i/2}\\ \zeta_{9-12} &= (\Omega_3+i\Omega_4 )^{1/4}e^{k\pi i/2}\\ \zeta_{13-16} &= (\Omega_3-i\Omega_4 )^{1/4}e^{k\pi i/2}\\ \end{align}\\ \text{for}\; k = 0,1,2,3. }\endtoggle $$
It is clear that $8$ of these roots ($\zeta_{1-8}\!$) are inside the contour.
Now consider $$I = \int_\gamma \! \frac{-4i}{x\sqrt[4]{P(x)}} \mathrm{d}x$$
We are now looking for a different way of evaluating this integral.
This integral will be split into many contours that include, the unit circle, small semi-circles around each root of $P(x)$ inside the unit circle, lines connecting the unit circle to the roots and lines connecting the roots to the origin. For a better idea of what contour I am describing, see the image at the bottom of the post.
It is good to note that the contribution from the small circles around each root is zero because the function tends to $\frac{1}{\sqrt[4]{z}}$ around the roots. Additionally, the total contribution of the lines connecting the unit circle to the root will be zero because it is the same path traversed in the opposite direction. This is not the case for the other lines because of the branch cut.
We can now write that the final result, $I$, is equal to the sum of the integrals that connect the roots to the origin. For a simplification,
$$\begin{align} \Omega_1 &= -\frac{2}{43}-\frac{i \sqrt{39}}{43}+\frac{1}{2} \sqrt{-\frac{7536}{1849}+\frac{16 i \sqrt{39}}{1849}}\\[.2cm] \xi &= \Omega_1^{1/4} \approx 0.8867082411 + 0.3793025249 i\end{align}$$
$\xi$ is one of the roots of $P(x)$. Using symmetry, which will be justified later, we find that
$$\large J = \int_0^1\!\!\! \frac{32 i}{t \sqrt[4]{\frac{86}{5}t^8\xi^8+\frac{86}{5 t^8 \xi^8}+\frac{16}{5} t^4\xi^4+\frac{16}{5 t^4 \xi^4}+36}} \mathrm{d}t$$
$$\large I = 12.01652200\ldots = \boxed{\Re({J}) + \Im({J})}$$
For numerical evidence
In :=
w= N[(-(2/43)-(I Sqrt[39])/43+1/2 Sqrt[-(7536/1849)+(16 I Sqrt[39])/1849])^(1/4),100]
J= NIntegrate[(32 I)/(t (36+86/(5 t^8 w^8)+16/(5 t^4 w^4)+16/5 t^4 w^4+86/5 t^8 w^8)^(1/4)),{t,0,1},WorkingPrecision->50];
Re[J]+Im[J]
Out := 12.016522007576859017247019246788042118792983861776
The next step is to find a simpler form for $J$. The most natural substitution is that of $x = t\xi$. With this, write
$$\begin{align} J &= 32 i\int_0^\xi\!\!\! \frac{ \mathrm{d}x}{x\sqrt[4]{\frac{86}{5}x^8+\frac{86}{5x^8}+\frac{16}{5} x^4+\frac{16}{5x^4}+36}} \\[.2cm] &= 32i \cdot 5^{1/4} \int_0^\xi \!\!\! \frac{x}{\sqrt[4]{86x^{16}+16x^{12}+180x^8+16x^4+86}}\mathrm{d}x\end{align}$$
This result may not seem satisfactory but I have turned the trigonometric integral into one involving only polynomials. This post is a work in progress so I may be able to find a nicer form for the integral in the coming time.
For the reasoning behind my choice of contour, see this picture. The $16$ roots are located at the tips of the white lines with the white lines being the branch cuts. Due to aforementioned simplification, the final integral is equal to the sum of the integrals along the white lines in the unit circle. Symmetry can be used to simplify the final evaluation.
Here is the code used to generate this image (stolen from this post).
GraphicsRow[Table[ContourPlot[g[f],{x,-1.5,1.5},{y,-1.5,1.5},ColorFunction->"GrayYellowTones",Epilog->{Cyan,Thick,Circle[{0,0}],Red,PointSize[0.02],Point[{Re@#,Im@#}&/@{}]},Contours->100],{g,{Re,Im}}]]
In addition to the several equivalent integrals already published, this one is of interest : $${\LARGE\int}_0^{\infty}\frac{4~dt}{\sqrt[{\LARGE 4}]{4A\cosh^2t+2B\cosh t+C-2A}}$$ Suppose that a closed form exists for this integral. By "closed form" I mean the combination of a limited number of standard functions. This closed form must be valid for particular values of $A,B,C$, for example $A=1/4, B=1/2, C=1/2$, i.e. the integral : $${\LARGE\int}_0^{\infty}\frac{dt}{\sqrt[{\LARGE 4}]{\cosh^2t+\cosh t}}$$ As far as I know, there is no closed form for it, which is in contradiction with the above supposition. That is the reason why I think that it is doubtful that a closed form exists for the first integral. May be, it might exists a closed form involving other special functions not referenced or not today considered as standard function in the publications.