Why does one define the polynomial ring $R[x] := R^{(\mathbb N)}$ and not as a subset of functions?

One reason this is important is that we would like to be able to distinguish between polynomial that have different coefficients but correspond to the same function. This comes up in the context of finite fields: for example, it is useful to distinguish between the polynomials $p(x) = x$ and $q(x) = x^3$, even though these two polynomials represent the same function over $\Bbb F_2$.

Outside the motivation of studying polynomials in and of themselves, it is important that polynomials are "domain-agnostic." Even though we are thinking of polynomials with coefficients from a ring $R$, it is not necessarily the case that the associated function of interest takes elements of $R$ as its inputs. In particular, it is often useful that the same polynomial describes not only a function over $R$, but also a function over a ring-extension $\bar R$, or even a function over an $R$-Algebra $A$.

As an example from linear algebra: given a matrix $M \in \Bbb F^{n \times n}$ and a polynomial $p \in \Bbb F[x]$, it is highly useful to be able to talk about the application $p(M)$. However, this is nonsense if we define a polynomial to be a function with domain $\Bbb F$.

This is because different polynomials can lead to the same function.

For example, take a finite ring. Clearly on a finite ring, there are finitely many functions of one variable (more exactly, if the ring has $n$ elements, then there are exactly $n^n$ different functions). But there are infinitely many different polynomials (for every $n\in\mathbb N$, $x^n$ is a polynomial different from any $x^m, m\ne n$).

Moreover, the same polynomial can lead to different polynomial functions. For example, take polynomials over the real numbers. Then for any real algebra $A$, the polynomials lead to functions $f:A\to A$ obtained by replacing the variable by an instance of $A$. Clearly that is something you cannot achieve easily given just functions $f:\mathbb R\to\mathbb R$.

For this post, we will live in the realm of commutative rings with unit.

Suppose you have a ring $R$. Naively, a polynomial with coefficients in $R$ is something that takes something (usually called $x$) as an input and gives you something in return. Usually, this something is an element of $R$ and you get an element of $R$ in return, but it turns out that it is convenient to leave this without saying, so that a polynomial can take elements of some set and turn it into an element of this same set.

However, since polynomial expressions include sums and products, the "domain" of the polynomial cannot be any set: it should be a ring. Moreover, the coefficients of the polynomial are elements of $R$, so the "domain" cannot be any ring: it also has to be a ring in which makes sense to multiply by elements of $R$. Technically, if you have a ring $S$, the idea of multiplication by elements of $R$ is encoded in the form of a ring homomorphism $\phi:R\to S$. Then for any $r\in R$ and $s\in S$ we can define the product $rs$ as $rs=\phi(r)s$. Such a pair $(S,\phi)$ is called an $R$-algebra.

Finally, we want to be able to perform operations on the polynomials even before evaluating them, including multiplication by elements of $R$. This means that our set of polynomials needs to be a ring and an $R$-algebra as well. Moreover, for any element $s\in S$ and any polynomials $p,q$, we want $(p+q)(s)=p(s)+q(s)$ and $(pq)(s)=p(s)q(s)$: that is, we want the evaluation $p\mapsto p(s)$ to be a ring homomorphism that respects the $R$-multiplication. Hence we get the following idea of a definition:

Given a ring $R$, we want to have and $R$-algebra called the polynomial ring $R[x]$ that satisfies the following property: for any $R$-algebra $S$ (this is, a ring in which "multiplication by elements of $R$ makes sense") and any element $s\in S$, we have an evaluation map $\operatorname{ev}_s:R[x]\to S$ which is a ring homomorphism respecting the $R$-multiplication, such that the polynomial $x\in R[x]$ gets mapped to $s\in S$. Moreover, the evaluation homomorphism should be unique.

It turns out that the usual definition of $R[x]$ as some formal sums and products satisfies this definition; even more: this $R[x]$ is the unique (in a suitable sense) $R$-algebra satisfying the definition.

For the more abstractly minded people, this post means that for any $R$-algebra $S$ there is a one-one correspondence

$$\left\{\text{elements of $S$}\right\} \leftrightarrow \left\{ \overset{ \displaystyle\text{ring homomorphisms} } { \underset{\textstyle\text{respecting $R$-multiplication}} {R[x]\to S} } \right\}$$

And for you categorists, the upshot is that the polynomial ring $R$ is the free commutative $R$-algebra on the one-element set.