How can we show that $\int_{0}^{\pi/2}x\cos(8x)\ln\left(1+\tan x\over 1-\tan x\right)\mathrm dx={\pi\over 12}?$

What you need is to split the integral into two parts and use integration by parts for the second part. Let $x\to\frac{\pi}{2}-x$ and then \begin{eqnarray} I&=:&\int_{0}^{\pi/2}x\cos(8x)\ln\left(1+\tan x\over 1-\tan x\right)\mathrm dx\\ &=&\int_{0}^{\pi/4}x\cos(8x)\ln\left(1+\tan x\over 1-\tan x\right)\mathrm dx+\int_{\pi/4}^{\pi/2}x\cos(8x)\ln\left(1+\tan x\over 1-\tan x\right)\mathrm dx\\ &=&\int_{0}^{\pi/4}x\cos(8x)\ln\left(1+\tan x\over 1-\tan x\right)\mathrm dx-\int_{\pi/4}^{0}(\frac{\pi}{2}-x)\cos(8x)\ln\left(1+\cot x\over 1-\cot x\right)\mathrm dx\\ &=&\int_{0}^{\pi/4}x\cos(8x)\ln\left(1+\tan x\over 1-\tan x\right)\mathrm dx+\int^{\pi/4}_{0}(\frac{\pi}{2}-x)\cos(8x)\ln\left(1+\tan x\over \tan x-1\right)\mathrm dx\\ &=&\int_{0}^{\pi/4}x\cos(8x)\ln\left(1+\tan x\over 1-\tan x\right)\mathrm dx+\int^{\pi/4}_{0}(\frac{\pi}{2}-x)\cos(8x)\left[\ln\left(1+\tan x\over 1-\tan x\right)+\pi i\right]\mathrm dx\\ &=&\int_{0}^{\pi/4}x\cos(8x)\ln\left(1+\tan x\over 1-\tan x\right)\mathrm dx+\int^{\pi/4}_{0}(\frac{\pi}{2}-x)\cos(8x)\ln\left(1+\tan x\over 1-\tan x\right)\mathrm dx\\ &&+\pi i\int^{\pi/4}_{0}(\frac{\pi}{2}-x)\cos(8x)\mathrm dx\\ &=&\frac{\pi}{2}\int^{\pi/4}_{0}\cos(8x)\ln\left(1+\tan x\over 1-\tan x\right)\mathrm dx\\ &=&\frac{\pi}{16}\int^{\pi/4}_{0}\ln\left(1+\tan x\over 1-\tan x\right)\mathrm d\sin(8x)\\ &=&\frac{\pi}{16}\left[\ln\left(1+\tan x\over 1-\tan x\right)\sin(8x)\bigg|_0^{\pi/4}-\int^{\pi/4}_{0}\sin(8x)\mathrm{d}\ln\left(1+\tan x\over 1-\tan x\right)\right]\\ &=&-\frac{\pi}{16}\int^{\pi/4}_{0}\sin(8x)\left(\frac1{1+\tan x}+\frac1{ 1-\tan x}\right)\sec^2x\mathrm d x\\ &=&-\frac{\pi}{8}\int^{\pi/4}_{0}\frac{\sin(8x)}{\cos(2x)}\mathrm d x\\ &=&\frac{\pi}{12}. \end{eqnarray} Here $$ \int^{\pi/4}_{0}(\frac{\pi}{2}-x)\cos(8x)\mathrm dx=0. $$

$\newcommand{\bbx}[1]{\,\bbox[8px,border:1px groove navy]{\displaystyle{#1}}\,} \newcommand{\braces}[1]{\left\lbrace\,{#1}\,\right\rbrace} \newcommand{\bracks}[1]{\left\lbrack\,{#1}\,\right\rbrack} \newcommand{\dd}{\mathrm{d}} \newcommand{\ds}[1]{\displaystyle{#1}} \newcommand{\expo}[1]{\,\mathrm{e}^{#1}\,} \newcommand{\ic}{\mathrm{i}} \newcommand{\mc}[1]{\mathcal{#1}} \newcommand{\mrm}[1]{\mathrm{#1}} \newcommand{\pars}[1]{\left(\,{#1}\,\right)} \newcommand{\partiald}[3][]{\frac{\partial^{#1} #2}{\partial #3^{#1}}} \newcommand{\root}[2][]{\,\sqrt[#1]{\,{#2}\,}\,} \newcommand{\totald}[3][]{\frac{\mathrm{d}^{#1} #2}{\mathrm{d} #3^{#1}}} \newcommand{\verts}[1]{\left\vert\,{#1}\,\right\vert}$

I'll assume the $\ds{\ln}$-argument is 'enclosed' in an absolute value !!!.

\begin{align} &\int_{0}^{\pi/2}x\cos\pars{8x} \ln\pars{\verts{1 + \tan\pars{x} \over 1 - \tan\pars{x}}}\,\dd x = \int_{0}^{\pi/2}x\cos\pars{8x} \ln\pars{\verts{\tan\pars{x + {\pi \over 4}}}}\,\dd x \\[5mm] = &\ -\int_{-\pi/4}^{\pi/4}\pars{x + {\pi \over 4}}\cos\pars{8x} \ln\pars{\verts{\tan\pars{x}}}\,\dd x = -\,{\pi \over 2}\int_{0}^{\pi/4}\cos\pars{8x} \ln\pars{\verts{\tan\pars{x}}}\,\dd x \\[5mm] = &\ {\pi \over 16}\int_{0}^{\pi/4}\sin\pars{8x} {\sec^{2}\pars{x} \over \tan\pars{x}}\,\dd x = {\pi \over 8}\int_{0}^{\pi/4}{\sin\pars{8x} \over \sin\pars{2x}}\,\dd x = {\pi \over 16}\,\Im\int_{0}^{\pi/2}{\expo{4\ic x} - 1 \over \sin\pars{x}}\,\dd x \\[5mm] = &\ \left.{\pi \over 16}\,\Im\int_{x\ =\ 0}^{x\ =\ \pi/2} {z^{4} - 1\over \pars{1 - z^{2}}\ic/\pars{2z}} \,{\dd z \over \ic z}\,\right\vert_{\ z\ =\ \exp\pars{\ic x}} = \left.{\pi \over 8}\,\Im\int_{x\ =\ 0}^{x\ =\ \pi/2}\pars{z^{2} + 1 } \,\dd z\,\right\vert_{\ z\ =\ \exp\pars{\ic x}} \\[1cm] \stackrel{\mrm{as}\ \epsilon\ \to\ 0^{+}}{\sim}&\ -\,{\pi \over 8}\,\Im\int_{1}^{0}\pars{-y^{2} + 1}\,\ic\,\dd y\ -\ \overbrace{% {\pi \over 8}\,\Im\int_{0}^{1}\pars{x^{2} + 1}\,\dd x} ^{\ds{=\ 0}} \\[5mm] - &\ {\pi \over 8}\,\Im\int_{\pi}^{\pi/2}\pars{\epsilon^{2}\expo{2\ic\theta} + 1}\epsilon\expo{\ic\theta}\ic\,\dd\theta \,\,\,\,\,\stackrel{\mrm{as}\ \epsilon\ \to\ 0^{+}}{\to}\,\,\,\,\, {\pi \over 8}\int_{0}^{1}\pars{1 - y^{2}}\,\dd y = \bbx{\ds{\pi \over 12}} \end{align}