Why can't a neighborhood be a finite set?

"Every finite set is closed" does not imply "every open neighbourhood is infinite".

Let $(X,d)$ be any metric space, and let $F \subseteq X$ be a non-empty finite subset; say $F = \{ x_1, \cdots, x_n \}$ for some $n \in \mathbb{N}$. To prove that $F$ is closed it suffices to prove that $X - F$ is open. So let $p \in X-F$. Then the set $\{ d(p,x_1), \cdots, d(p,x_n) \}$ has a minimum value, say $\delta$, and we know that $\delta > 0$ since $p \not \in F$, and so whenever $d(p,q) < \delta$ we have $q \in X-F$. But this tells you that the (open) ball of radius $\delta$ about $p$ lies in $X-F$, and hence $X-F$ is open.

In your examples, yes, the sets are open and finite, but they are also closed!

The key fact to take home is that sets are allowed to be both open and closed.

---

However, there is a partial converse: in a dense metric space, every open neighbourhood is infinite.

A metric space $(X,d)$ is dense if for any $x \in X$ and $r > 0$ there exists $y \in X$ with $x \ne y$ and $d(x,y) < r$ $-$ that is, given any point in the space, there are other points which lie arbitrarily close to that point. $\mathbb{R}$ is a dense space, so is $\mathbb{Q}$, so is $(0,1)$, and indeed so is any other non-empty open subset of $\mathbb{R}$.

The fact that open neighbourhoods in dense metric spaces are necessarily infinite is clear from the definition. [It's also clear that a metric space in which every open neighbourhood is infinite is dense, and so the two conditions are equivalent.]

For an interesting example, let $X$ be any non-empty set, and define $d:X\times X\to\Bbb R$ by $$d(x,y)=\begin{cases}0 & x=y\\1 & x\neq y.\end{cases}$$ This can be shown to be a metric on $X$ (called the discrete metric). One interesting property is that in the metric space $(X,d)$, every subset of $X$ is both open and closed. Even in general metric spaces, there will always be at least two subsets that are both open and closed--namely, the empty set and the whole set.