Motivation for zeta function of an algebraic variety
The definition using exponential of such an ad hoc looking series is admittedly not too illuminating. You mention that the series looks vaguely logarithmic, and that's true because of denominator $m$. But then we can ask, why include $m$ in the denominator?
A "better" definition of a zeta function of a curve (more generally a variety) over $\mathbb F_p$ involves an Euler product. The product will be over all points $P$ of $V$ which are defined over the algebraic closure $\overline{\mathbb F_p}$ (this isn't exactly true, see below). Any such point has a minimal field of definition, namely the field $\mathbb F_{p^n}$ generated by the coordinates of this point. We shall define the norm of this point $P$ as $|P|=p^n$. Then we can define $$\zeta_{V,p}(s)=\prod_P(1-|P|^{-s})^{-1}.$$ (again, this is not quite right) Why would this definition be equivalent to yours? It's easiest to see by taking the logarithms. Then for a point $P$, the logarithm of the corresponding factor of the product will contribute $$-\log(1-|P|^{-s})=\sum_{k=1}^\infty\frac{1}{k}|P|^{-ks}=\sum_{k=1}^\infty\frac{1}{k}p^{-nks}=\sum_{k=1}^\infty\frac{n}{nk}(p^{nk})^{-s}.$$ In the last step I have multiplied the numerator and the denominator by $n$, because the point $P$ contributes precisely to numbers $N_{nk}$, since $P$ is defined over all the fields $\mathbb F_{p^{nk}}$.
But we see a problem - this way, we have counted each point $n$ times because of $n$ in the numerator. The resolution is rather tricky - instead of taking a product over points, we take a product over Galois orbits of the points - if we have a point $P$ minimally defined over $\mathbb F_{p^n}$, then there are exactly $n$ points (conjugates of $P$) which we can reach from $P$ by considering the automorphisms of $\mathbb F_{p^n}$. If we were to write $Q$ for this set of conjugates, and we define $|Q|=p^n$, then repeating the calculation above we see that we always count $Q$ $n$ times - which is just right, since it consists of $n$ points! Thus we arrive at the following (this time correct) definition of the zeta function: $$\zeta_{V,p}(s)=\prod_Q(1-|Q|^{-s})^{-1},$$ the product this time over Galois orbits.
Apart from being (in my opinion) much better motivated, it has other advantages. For instance, from the product formula it is clear that the series has integer coefficients. Further, it highlights the similarity with the Riemann zeta function, which has a very similar Euler product. Both of those are generalized to the case of certain arithmetic schemes, but that might be a story for a different time.
As for your last question, regarding rationality, this is a rather nontrivial result, even, as far as I know, for curves. If you are interested in details, then I recommend taking a look at Koblitz's book "$p$-adic numbers, $p$-adic analysis and zeta functions". There he proves, using moderately elementary $p$-adic analysis, rationality of zeta functions of arbitrary varieties.
As KConrad says in the comment, the proof of rationality actually is much simpler for curves than for general varieties. I imagine it is by far more illuminating than Dwork's proof as presented in Koblitz.
Exercise 4.8 of Enumerative Combinatorics, vol. 1, second ed., and Exercise 5.2(b) in volume 2 give an explanation of sorts for general varieties over finite fields. According to Exercise 4.8, a generating function $\exp \sum_{n\geq 1} a_n\frac{x^n}{n}$ is rational if and only if we can write $$ a_n=\sum_{i=1}^r\alpha_i^n-\sum_{j=1}^s \beta_j^n, $$ for nonzero complex numbers $\alpha_i$, $\beta_j$ (independent of $n$). This is stronger than saying that $\sum_{n \geq 1}a_nx^n$ is rational. Moreover, if the variety $V$ is defined over $\mathbb{F}_q$ and $N_n$ is the number of points over $\mathbb{F}_{q^n}$, then the solution to Exercise 5.2(b) is a simple argument showing that $\exp \sum_{n\geq 1}N_n\frac{x^n}{n}$ has integer coefficients. It corresponds to partitioning the rational points into their Galois orbits.