$-1 = 0$ by integration by parts of $\tan(x)$

Without even reading I answer: the antiderivatives of a function are equal only up to a an additive constant, that is any two antiderivatives will always differ by a constant on an interval.

Edit: Ok, having now read the question I confirm my suspicion, note that the symbol $\int f(x)dx$ is not a well defined function. You should interpret the symbol $\int f(x)dx$ as being an undertermined differentiable function which, once you differentiate, yields $f(x)$. Although more formally I believe it's more common to define $\int f(x)dx$ as the set of functions described above, that is, for some non degenerate interval $I$, $$\int f(x)dx=\{F\in \Bbb R^I: \text{F is differentiable and }(\forall x\in I)(F'(x)=f(x))\}.$$

Using this definition one has to very careful about what one means with $\int f(x)dx=g(x)$, because it doesn't mean what one would initially suspect.

Let's try using "real" primitives. Suppose $-\pi/2<x<\pi/2$; then \begin{align} \int_0^x \tan t\,dt &=\left[(-\cos t)\frac{1}{\cos t}\right]_0^x -\int_0^x (-\cos t)\frac{\sin t}{\cos^2 t}\,dt\\ &=\bigl[-1\bigr]_0^x+\int_0^x\tan t\,dt\\ &=(-1)-(-1)+\int_0^x\tan t\,dt\\ &=\int_0^x\tan t\,dt \end{align} which of course doesn't say much, does it? ;-)

Fixing the lower limit of integration is choosing a well determined primitive (or antiderivative). Always do it in case of doubt.