Relationships between bounded and convergent series
Whenever we have a series, $$\sum_{i=1}^{\infty} a_i,$$ we "automatically" get two sequences out of that series:
- The sequence of terms, which is $a_1,a_2,a_3,\ldots$; and
- The sequence of partial sums, which is $s_1,s_2,s_3,\ldots$, where $$\begin{align*} s_1 &= a_1\\ s_2 &= a_1+a_2\\ s_3 &= a_1+a_2+a_3\\ &\vdots\\ s_n &= \sum_{i=1}^n a_i = a_1+a_2+\cdots + a_n. \end{align*}$$
When we talk about "convergence of the series", we are really talking about convergence of the sequence of partial sums: the series $\sum a_i$ converges if and only if the sequence $(s_n)$ converges. That is, your definitions about "series" are really about "sequence of partial sums", and so you have the usual relationship:
In particular, $$\sum_{i=1}^{\infty}a_i\text{ converges}\Longleftrightarrow \{s_i\}_{i=1}^{\infty}\text{ converges}\Longrightarrow \{s_i\}_{i=1}^{\infty}\text{ is bounded}\Longleftrightarrow \sum_{i=1}^{\infty}a_i\text{ is bounded}$$ (where "is bounded" is as per your definition above); but it is possible for $\{s_i\}_{i=1}^{\infty}$ to be bounded, and not convergent, so one can have a series $\sum_{i=1}^{\infty}a_i$ that is bounded (i.e., the sequence of partial sums is bounded) but does not converge.
A simple example of this is $\sum_{i=1}^{\infty} (-1)^n$. The partial sums are $s_{2k+1} = -1$ and $s_{2k}=0$ for every $k$, so the sequence of partial sums is: $$-1,\ 0,\ -1,\ 0,\ -1,\ldots$$ which is bounded but not convergent. So the series is bounded but not convergent.
The relevant theorem for sequences, as you are no doubt aware, is:
Theorem. If $\{b_n\}$ is a monotone sequence, then $\{b_n\}$ converges if and only if it is bounded.
How does that translate for series? When is the sequence of partial sums monotone?
$\{s_i\}$ is increasing if and only if $s_n\leq s_{n+1}$ for all $n$, if and only if $s_{n+1}-s_n\geq 0$ for all $n$; but $s_{n+1}-s_n = a_{n+1}$. So:
The sequence of partial sums of $\displaystyle \sum_{i=1}^{\infty}a_i$ is increasing if and only if all the terms $a_i$ are nonnegative. The sequence of partials sums is strictly increasing if and only if all the terms $a_i$ are positive.
Likewise,
The sequence of partial sums of $\displaystyle \sum_{i=1}^{\infty}a_i$ is decreasing if and only if all the terms $a_i$ are nonpositive. The sequence of partial sums is strictly decreasing if and only if all the terms $a_i$ are negative.
So we conclude:
Theorem. Let $\displaystyle \sum_{i=1}^{\infty}a_i$ is a series in which every term $a_i$ is nonnegative. Then the series converges if and only if it is bounded (in the sense that the sequence of partial sums is bounded).
No, a bounded series does not necessarily converge. Consider the series $\displaystyle \sum (-1)^n $ (heavily related to Henning's example). It will forever oscillate between 0 and 1 (or -1 and 0, depending on the indices).
But if the partial sums are bounded and monotonic, then it does converge.
But in either case, it's a bit weaker than the converse - convergent series always have bounded partial sums.
A convergent sequence is bounded, but a bounded sequence is not necessarily convergent. Consider, for example the sequence (1, -1, 1, -1, 1, -1, ...).
On the other hand, an increasing (or decreasing) bounded sequence in $\mathbb R$ will necessarily converge.