Looking for a closed form for $\sum_{k=1}^{\infty}\left( \zeta(2k)-\beta(2k)\right)$

There is a closed form for your series:

$$ \sum_{k=1}^{\infty}\left( \zeta(2k)-\beta(2k)\right)=\frac{1}{2}+\frac{1}{2}\ln2. \tag1 $$

Proof. Using absolute convergence of the series, you may write

$$\begin{align} \sum_{k=1}^{\infty}\left( \zeta(2k)-\beta(2k)\right) & = \sum_{k=1}^{\infty}\left( \sum_{n=1}^{\infty}\frac{1}{n^{2k}}-\sum_{n=1}^{\infty}\frac{(-1)^{n-1}}{(2n-1)^{2k}}\right)\\\\ & = \sum_{k=1}^{\infty}\left( \sum_{n=\color{red}2}^{\infty}\frac{1}{n^{2k}}-\sum_{n=\color{red}2}^{\infty}\frac{(-1)^{n-1}}{(2n-1)^{2k}}\right)\\\\ & = \sum_{n=2}^{\infty}\left( \sum_{k=1}^{\infty}\frac{1}{n^{2k}}-\sum_{k=1}^{\infty}\frac{(-1)^{n-1}}{(2n-1)^{2k}}\right)\\\\ & = \sum_{n=2}^{\infty}\left( \frac{1}{n^{2}}\frac{1}{1-\frac{1}{n^{2}}}+\frac{(-1)^{n}}{(2n-1)^{2}}\frac{1}{1-\frac{1}{(2n-1)^{2}}}\right)\\\\ & = \sum_{n=2}^{\infty}\frac{1}{(n+1)(n-1)}+\sum_{n=2}^{\infty}\frac{1}{8n(2n-1)}-\sum_{n=2}^{\infty}\frac{1}{8(2n-1)(n-1)}\\\\ & = \frac{1}{2}+\frac{1}{2}\ln2, \end{align}$$ where, in the last steps, we have used partial fraction decomposition, telescoping terms and a familiar series for $\ln 2$.

Hint: Rewrite the general term as an infinite series, and then switch the order of summation. :-$)$