example of a set that is closed and bounded but not compact

If $\{e_n\}$ is an (infinite) orthonormal set in a Hilbert space $H$ then $\{e_1,e_2,...\}$ is closed and bounded but not compact. It is not compact because there is no convergent subsequence (since $\|e_n-e_m\|=\sqrt2$ for $n \neq m)$.

You're on the right track. If we consider $X=\left\{\frac1n:n\in\Bbb N^+\right\}$ in the discrete topology, then we can endow it with the metric $d:X\times X\to\Bbb R$ given by $$d(x,y)=\begin{cases}0 & x=y\\1 & \text{otherwise,}\end{cases}$$ which does indeed induce the discrete topology on $X$ (it's called the discrete metric for this reason). Then $X$ is certainly bounded, as any ball of radius greater than $1$ necessarily includes the whole set, and is certainly closed in itself (as all spaces are). However, it is not compact, since the open cover by singletons admits no finite subcover, as you've observed. More generally, any infinite discrete space admits a proper subspace that is closed and bounded, but not compact (delete any point).

We could come to the same conclusions if we considered $X$ as a space under the metric $$\rho(x,y)=|x-y|.$$ Indeed, $\rho$ induces the discrete topology on $X$, as well, and we similarly find that $X$ is bounded under $\rho$.

The kicker, here, is the boundedness. You need to specify a metric, or some other convention to determine boundedness, not just a topology. For example, $\Bbb Z$ considered as a subspace of $\Bbb R$ is indeed discrete, but while it is bounded in the discrete metric, it is not bounded in the standard metric on $\Bbb R$.

The "closed" ball $\lVert x \rVert \leq 1$ in any infinite dimensional Banach space is closed and bounded but not compact. It is closed because any point outside it is contained in a small open ball disjoint from the first one, by the triangle inequality. That is, if $\lVert y \rVert = 1 + 2 \delta,$ then the sets $\lVert x \rVert \leq 1$ and $\lVert x - y \rVert < \delta$ are disjoint. Hence the complement of the "closed" unit ball is open and the "closed" unit ball really is closed. But not compact if not in finite dimensions.