If $G$ has only 2 proper, non-trivial subgroups then $G$ is cyclic
Let $H_1$ and $H_2$ be the two non-trivial proper subgroups of the given group $G$. I claim that $G$ is not the union $H_1\cup H_2$. If one of the subgroups is contained in the other, then this is trivially true. Otherwise there exist elements belonging to one subgroup but not the other. Let $h_1\in H_1\setminus H_2$ and $h_2\in H_2\setminus H_1$. What about $g=h_1h_2$? If it belongs to $H_1$, then so does $h_2$. If it belongs to $H_2$, then so does $h_1$. In either case we contradict our assumptions, so we have to conclude that $g\notin H_1\cup H_2$.
So we know that there exists an element $g\in G$, $g\notin H_1\cup H_2$. What is the subgroup generated by $g$? Can't be either $H_1$ or $H_2$, so it has to be all of $G$. Ergo, $G$ is cyclic.
First note that if $G$ does not have finite order, then it does not have a finite number of subgroups, so we can assume that $G$ is finite (see the comment by Pete Clark).
Note that if $3$ distinct primes divides the order of the group, then the group has at least $3$ proper non-trivial subgroups.
So $|G| = p^nq^m$ with $p$ and $q$ primes. Now, if either $n$ or $m$ is greater than or equal to $4$, then the corresponding Sylow subgroup has too many subgroups. Also, if either if at least $2$ and the other is not $0$, we again get too many subgroups.
We are left with either $|G| = p^3$ or $|G| = pq$. In both cases the cyclic group of that order will satisfy the conditions, and we wish to show that these are the only ones (since the cyclic group of order $p^2$ has too few subgroups, and the non-cyclic one has too many).
If $|G| = pq$ and $G$ is not cyclic, then $G$ is not abelian, and thus has more than one Sylow subgroup for either $p$ or $q$, giving us too many subgroups.
If $|G| = p^3$ then $G$ has at least one subgroup of order $p$ and one of order $p^2$. But if $G$ is not cyclic, it has more than one maximal subgroup, which gives us at least two of order $p^2$, again resulting in too many subgroups.
Since $G$ has proper non-trivial subgroups $\exists~a~(\neq e)\in G.$
Case $1$: $G=(a):$ Nothing left to prove.
Case $2$: $(a)$ is a non-trivial proper subgroup of $G:$ Choose $b\in G-(a).$
Case $2.1:$ $G=(b):$ Nothing left to prove.
Case $2.2:$ $(b)$ is also a non-trivial proper subgroup of $G:$
Case $2.2.1:$ $(a)\cup(b)=G,$ a subgroup of $G.$
Consequently either $(a)\subset(b)$ or $(b)\subset(a).$
Case $2.2.2:$ $\exists~c\in G-(a)\cup(b).$
Since $G$ has only two proper subgroups $G=(c).$