Is every series that matters a Taylor series?

How about the Riemann zeta function? $$ \zeta(s):=\sum_{n\in\mathbb N}n^{-s}\qquad \operatorname{re} s>1 $$

This is manifestly not a Taylor series, yet it is analytic on its domain of definition (and can be continued to all $\mathbb C\setminus\{0\}$).

See also Laurent series, Puiseux series, etc.

There are also the series $$\sum_{(a,b) \in \mathbb{Z}^2 \setminus\{(0,0)\}} \frac{1}{(a\tau + b)^{2k}},$$ where $k \geq 2$. Though these converge on the open set Im$(\tau) > 0$. They are important in the theories of modular forms and elliptic functions. They satisfy infinitely many symmetries !