Why does the $\sum_{n>1}(\zeta(n)-1)=1?$
Note: $$\begin{align}\sum_{n=2}^\infty (\zeta(n)-1) &= \sum_{n=2}^\infty \sum_{k=2}^\infty\frac{1}{k^n}\\ &=\sum_{k=2}^\infty\sum_{n=2}^\infty \frac{1}{k^n} \end{align}$$
And $$\sum_{n=2}^{\infty} \frac{1}{k^n} =\frac{1}{k^2}\frac{1}{1-\frac{1}{k}}= \frac{1}{k(k-1)}=\frac{1}{k-1}-\frac{1}{k}.$$
For $\zeta(2n)$ case:
$$ \sum_{n=1}^{\infty} \frac{1}{k^{2n}} = \frac{1}{k^2-1} = \frac{1}{2}\left(\frac{1}{k-1}-\frac{1}{k+1}\right)$$
More generally, if $f(z)=\sum_{n=2}^\infty a_nz^n$ has radius of convergence more than $\frac{1}2$, then:
$$\sum_{n=2}^\infty a_n(\zeta(n)-1) = \sum_{k=2}^\infty f\left(\frac1k\right)$$
This can be used to show that $$\sum_{n=2}^\infty \frac{\zeta(n)-1}{n} = 1-\gamma$$ where $\gamma$ is the Euler–Mascheroni constant. Using the standard limit for $\gamma$, we see that:
$$\lim_{N\to\infty} \left(\log N -\sum_{n=2}^N \frac{\zeta(n)}{n}\right) = 0$$
Very late comment
I just noticed that if $f(z)=\sum a_nz^n$ has radius of convergence greater than $1,$ we get:
$$\sum_{n=2}^\infty a_n \zeta(n) = \sum_{k=1}^\infty f\left(\frac 1k\right)$$
Using the series representation of the Riemann-Zeta function
$$\zeta(n)=\sum_{k=1}^{\infty}\frac{1}{k^n}$$
gives
$$\begin{align} \sum_{n=2}^{\infty}\left(\sum_{k=1}^{\infty}\frac{1}{k^n}-1\right)&=\sum_{k=2}^{\infty}\sum_{n=2}^{\infty}\frac{1}{k^n}\\\\ &=\sum_{k=2}^{\infty}\left(\frac{1/k^2}{1-1/k}\right)\\\\ &=\sum_{k=2}^{\infty}\left(\frac{1}{k-1}-\frac{1}{k}\right)\\\\ &=1 \end{align}$$
There is a whole series of formulae like this. The proofs are all along the lines of writing out the $\zeta$ sums, changing the order of summation (making sure that this is valid, of course!), and doing the interior sum. In this case, the double sum will be $$ \begin{align} \sum_{n=2}^{\infty} (\zeta(n)-1) &= \sum_{n=2}^{\infty} \sum_{k=2}^{\infty} \frac{1}{k^n} \\ &= \sum_{k=2}^{\infty} \sum_{n=2}^{\infty} \frac{1}{k^n} \\ &= \sum_{k=2}^{\infty} \frac{1}{k^2(1-1/k)} \\ &= \sum_{k=2}^{\infty} \frac{1}{k(k-1)} \\ &= \sum_{k=2}^{\infty} \left( \frac{1}{k} - \frac{1}{k-1} \right), \end{align} $$ which it is easy to see telescopes down to $1$. Some other examples can be found here.