How to calculate $\int_0^{\frac{\pi}{2}}{x^2\cot x\ln\cos x\, \text{d}x}$.
$$\color{blue}{\int_0^{\pi /2} {{x^2}\cot x\ln (\cos x)\mathrm{d}x} = - \frac{{{\pi ^4}}}{{720}} + \frac{{\ln ^42}}{{24}} - \frac{{{\pi ^2}\ln ^22}}{6} + \text{Li}_4\left(\frac{1}{2}\right)}$$
It is easy to show that (using Fourier expansion of $\ln(\cos x)$ for instance): $$\int_0^{\pi/2} x^2 \cot x \ \mathrm{d}x = \frac{\pi^2 \ln 2}{4} - \frac{7\zeta(3)}{8}$$ Hence it suffices to consider $$I = \int_0^{\pi /2} {x^2}\cot x\ln (2\cos x)\mathrm{d}x $$ this alternative form will be proved convenient.
Now consider the twins: $$P = \int_0^1 {\frac{{{{\ln }^2}x\ln (1 + x)}}{{1 - x}}\mathrm{d}x} \qquad Q = \int_0^1 {\frac{{{{\ln }^2}x\ln (1 - x)}}{{1 + x}}\mathrm{d}x} $$ We will see that $I$ arises from integrating $\ln^2 z \ln(1+z) / (1-z) $ around the semicircle contour above real axis. Indeed $$ P + \int_{ - 1}^0 {\frac{{{{\ln }^2}x\ln (1 + x)}}{{1 - x}}\mathrm{d}x} + \int_0^\pi {i{e^{ix}}\frac{{{{\ln }^2}({e^{ix}})\ln (1 + {e^{ix}})}}{{1 - {e^{ix}}}}\mathrm{d}x} = 0 $$ Hence $$\begin{aligned} P + \int_0^1 {\frac{{{{(\ln x + \pi i)}^2}\ln (1 - x)}}{{1 + x}}\mathrm{d}x} &= i\int_0^\pi {{x^2}\frac{{{e^{ix}}}}{{1 - {e^{ix}}}}\ln (1 + {e^{ix}})\mathrm{d}x} \\ &= - 4\int_0^{\pi/2} {{x^2}(\cot x + i)\left[ {\ln (2\cos x) + ix } \right]\mathrm{d}x} \end{aligned}$$ Discard imaginary part: $$P + Q - {\pi ^2}\int_0^1 {\frac{{\ln (1 - x)}}{{1 + x}}\mathrm{d}x} = - 4\int_0^{\pi /2} {\left[ {{x^2}\cot x\ln (2\cos x) - {x^3}} \right]\mathrm{d}x} = -4I + \frac{\pi^4}{16}$$
Thus it suffices to find $P$ and $Q$. If you're familiar with Euler sum or polylogarithm, these logarithm integrals of weight $4$ are standard. But nevertheless I will delineate how they're obtained. $$\begin{aligned} P &= \int_0^1 {\frac{{{{\ln }^2}x\ln (1 - {x^2})}}{{1 - x}}\mathrm{d}x} - \int_0^1 {\frac{{{{\ln }^2}x\ln (1 - x)}}{{1 - x}}\mathrm{d}x} \\ &= \int_0^1 {\left( {\frac{x}{{1 - {x^2}}} + \frac{1}{{1 - {x^2}}}} \right){{\ln }^2}x\ln (1 - {x^2})\mathrm{d}x} - \int_0^1 {\frac{{{{\ln }^2}x\ln (1 - x)}}{{1 - x}}\mathrm{d}x} \\ &= - \frac{7}{8}\int_0^1 {\frac{{{{\ln }^2}x\ln (1 - x)}}{{1 - x}}\mathrm{d}x} + \int_0^1 {\frac{{{{\ln }^2}x\ln (1 - {x^2})}}{{1 - {x^2}}}\mathrm{d}x} \\ &= - \frac{7}{8}\int_0^1 {\frac{{{{\ln }^2}x\ln (1 - x)}}{{1 - x}}\mathrm{d}x} + \frac{1}{8}\int_0^1 {{x^{ - 1/2}}\frac{{{{\ln }^2}x\ln (1 - x)}}{{1 - x}}\mathrm{d}x} \end{aligned}$$ Thus the value of $P$ can be calculated from the partial derivatives of $$\int_0^1 x^{a-1} (1-x)^{b-1} \mathrm{d}x = \frac{\Gamma(a)\Gamma(b)}{\Gamma(a+b)}$$ and the result is $$P = -\frac{19\pi^4}{720} + \frac{7}{2}\ln 2 \zeta(3)$$
$Q$ does not succumb under such line of attack, indeed: $$Q = \frac{\pi^4}{90} + \frac{\pi^2 \ln^2 2}{6} - \frac{\ln^4 2}{6} - 4 \text{Li}_4 \left(\frac{1}{2}\right) $$
Denote $$A= \int_0^1 \frac{\ln x \ln^2(1-x)}{x} \mathrm{d}x \qquad B= \int_0^1 \frac{\ln x \ln^2(1+x)}{x} \mathrm{d}x \qquad C= \int_0^1 \frac{\ln x \ln(1+x) \ln(1-x)}{x} \mathrm{d}x $$ Due to the Euler sum $\sum H_n / n^3 = \pi^4 / 72$ (or via beta function as above), $A = -\pi^4 / 180$. Integration by parts on $C$ gives $$2C = P-Q$$ Also $$A + B + 2C = \int_0^1 {\frac{{\ln x{{\ln }^2}(1 - {x^2})}}{x}\mathrm{d}x} = \frac{1}{4}\int_0^1 {\frac{{\ln x{{\ln }^2}(1 - x)}}{x}\mathrm{d}x} = \frac{A}{4}$$ The reason for introducing these integrals is that $B$ is tamer than $Q$, its indefinite integral is a simple combination of polylogarithms up to order $4$, from which you can calculate: $$B = \frac{\pi^4}{24}+ \frac{\pi^2 \ln^2 2}{6} - \frac{\ln^4 2}{6} - 4\text{Li}_4(\frac{1}{2}) - \frac{7}{2}\ln 2 \zeta(3)$$
From these you can find the value of $Q$, hence finally $I$.
In the book, Almost Impossible Integrals, Sums and Series, page $247$ Eq $3.288$ we have
$$\cot x\ln(\cos x)=\sum_{n=1}^\infty(-1)^n\left(\psi\left(\frac{n+1}{2}\right)-\psi\left(\frac{n}{2}\right)-\frac1n\right)\sin(2nx)$$
$$=\sum_{n=1}^\infty(-1)^n\left(\int_0^1\frac{1-t}{1+t}t^{n-1}dt\right)\sin(2nx),\quad 0<x<\frac{\pi}{2}$$
Thus,
$$\int_0^{\pi/2}x^2\cot x\ln(\cos x)dx=\sum_{n=1}^\infty(-1)^n\left(\int_0^1\frac{1-t}{1+t}t^{n-1}dt\right)\left(\int_0^{\pi/2}x^2\sin(2nx)dx\right)$$
$$=\sum_{n=1}^\infty(-1)^n\left(\int_0^1\frac{1-t}{1+t}t^{n-1}dt\right)\left(\frac{\cos(n\pi)}{4n^3}-\frac{3\zeta(2)\cos(n\pi)}{4n}-\frac{1}{4n^3}\right)$$
$$=\sum_{n=1}^\infty(-1)^n\left(\int_0^1\frac{1-t}{1+t}t^{n-1}dt\right)\left(\frac{(-1)^n}{4n^3}-\frac{3\zeta(2)(-1)^n}{4n}-\frac{1}{4n^3}\right)$$
$$=\frac14\int_0^1\frac{1-t}{t(1+t)}\left(\sum_{n=1}^\infty\frac{t^n}{n^3}-\frac{3\zeta(2)t^n}{n}-\frac{(-t)^n}{n^3}\right)dt$$
$$=\frac14\int_0^1\left(\frac1t-\frac2{1+t}\right)\left(\text{Li}_3(t)+3\zeta(2)\ln(1-t)-\text{Li}_3(-t)\right)dt$$
$$=\frac14\underbrace{\int_0^1\frac{\text{Li}_3(t)-\text{Li}_3(-t)}{t}dt}_{\mathcal{I}_1}-\frac12\underbrace{\int_0^1\frac{\text{Li}_3(t)-\text{Li}_3(-t)}{1+t}dt}_{\mathcal{I}_2}$$ $$+\frac34\zeta(2)\underbrace{\int_0^1\frac{\ln(1-t)}{t}dt}_{\mathcal{I}_3}-\frac32\zeta(2)\underbrace{\int_0^1\frac{\ln(1-t)}{1+t}dt}_{\mathcal{I}_4}$$
$$\mathcal{I}_1=\text{Li}_4(1)-\text{Li}_4(-1)=\zeta(4)+\frac78\zeta(4)=\boxed{\frac{15}{8}\zeta(4)}$$
By integration by parts we have
$$\mathcal{I}_2=\frac74\ln(2)\zeta(3)-\int_0^1\frac{\ln(1+t)\text{Li}_2(t)}{t}dt+\int_0^1\frac{\ln(1+t)\text{Li}_2(-t)}{t}dt$$
$$=\frac74\ln(2)\zeta(3)+\sum_{n=1}^\infty\frac{(-1)^n}{n}\int_0^1 t^{n-1}\text{Li}_2(t)dt-\frac12\text{Li}_2^2(-t)|_0^1$$
$$=\frac74\ln(2)\zeta(3)+\sum_{n=1}^\infty\frac{(-1)^n}{n} \left(\frac{\zeta(2)}{n}-\frac{H_n}{n^2}\right)-\frac5{16}\zeta(4)$$ $$=\frac74\ln(2)\zeta(3)-\frac54\zeta(4)-\sum_{n=1}^\infty\frac{(-1)^nH_n}{n^3}-\frac5{16}\zeta(4)$$ substitute
$$\sum_{n=1}^\infty\frac{(-1)^nH_n}{n^3}=2\operatorname{Li_4}\left(\frac12\right)-\frac{11}4\zeta(4)+\frac74\ln2\zeta(3)-\frac12\ln^22\zeta(2)+\frac{1}{12}\ln^42$$
we get
$$\mathcal{I}_2=\boxed{-2\operatorname{Li_4}\left(\frac12\right)-\frac{25}{16}\zeta(4)+\frac12\ln^22\zeta(2)-\frac{1}{12}\ln^42}$$
$$\mathcal{I}_3=-\text{Li}_2(1)=\boxed{-\zeta(2)}$$
$$\mathcal{I}_4=\int_0^1\frac{\ln(1-t)}{1+t}dt=\int_0^1\frac{\ln x}{2-x}dx=\sum_{n=1}^\infty\frac1{2^n}\int_0^1 x^{n-1}\ln xdx$$ $$=-\sum_{n=1}^\infty\frac{1}{n^22^n}=-\text{Li}_2\left(\frac12\right)=\boxed{\frac12\ln^22-\frac12\zeta(2)}$$
Combine all boxed results we finally get
$$\int_0^{\pi/2}x^2\cot x\ln(\cos x)dx=\text{Li}_4\left(\frac12\right)-\frac18\zeta(4)-\ln^2(2)\zeta(2)+\frac{1}{24}\ln^4(2)$$
By Fourier series $$ \log\cos x=\sum_{n\geq 1}\frac{1-\cos(2nx)}{n}(-1)^n \tag{A}$$ $$ \int_{0}^{\pi/2}x^2\cot(x)\log\cos(x)\,dx\\ = \int_{0}^{\pi/2}\left(\frac{\pi^2}{12}+\sum_{n\geq 1}\frac{(-1)^n \cos(2nx)}{n^2}\right)\sum_{n\geq 1}\frac{\cos(x)-\cos(2nx)\cos(x)}{n\sin(x)}(-1)^n\,dx\tag{B}$$ so the problem boils down to the evaluation of some alternating Euler sums.