In a compact lie group, can two closed connected subgroups generate a non-closed subgroup?

The abstract subgroup generated by $H$ and $K$ is closed.

We may assume that $G$ is connected. The groups $G$, $H$, $K$ are the groups of real points of real algebraic groups $\mathbf{G}$, $\mathbf{H}$, $\mathbf{K}$. Let $\frak g$, $\frak h$ , $\frak k$ be their Lie algebras. The complexifications ${\frak g}_{\Bbb C}$, ${\frak h}_{\Bbb C}$, ${\frak k}_{\Bbb C}$ are algebraic Lie algebras, i.e. Lie algebras of complex algebraic groups. Let ${\frak l}_{\Bbb C}$ be the Lie subalgebra of ${\frak g}_{\Bbb C}$ generated by Lie subalgebras ${\frak h}_{\Bbb C}$ and ${\frak k}_{\Bbb C}$. We refer to the book: Onishchik, A. L.; Vinberg, È. B.: Lie groups and algebraic groups, Springer-Verlag, Berlin, 1990. By Theorem 3.3.2 of this book, the Lie subalgebra ${\frak l}_{\Bbb C}$ is algebraic, i.e. it is the Lie algebra of a unique connected complex algebraic subgroup $\mathbf{L}_{\Bbb C}\subset \mathbf{G}_{\Bbb C}$. Clearly $\mathbf{L}_{\Bbb C}$ is defined over $\Bbb R$, i.e. comes from some real algebraic subgroup $\mathbf{L}\subset \mathbf{G}$. Set $L=\mathbf{L}(\Bbb R)$. Since $\mathbf{L}$ is connected and compact, the group of real points $L$ is connected, see Onishchik and Vinberg, Corollary 1 of Theorem 5.2.1. The Lie algebra $\frak l$ of $L$ is generated by the subalgebras $\frak h$ and $\frak k$. Since $H$ and $K$ are connected, $\mathbf{L}$ contains $\mathbf{H}$ and $\mathbf{K}$, and $L$ contains $H$ and $K$.

Let $L'$ denote the abstract subgroup generated by $H$ and $K$, it is contained in $L$. Since the Lie algebra $\frak l$ is generated by $\frak h$ and $\frak k$, one can easily see that for any element $A\in \frak l$ there exists a smooth map $\varphi$ from an interval $(-\varepsilon, \varepsilon)$ to $L$ with image contained in $L'$ and such that $d\varphi|_0=A$. It follows that $L'$ contains an open neighborhood of $1$ in $L$. Since $L$ is connected, we see that $L'=L$. Thus the abstract subgroup $L'$ generated by $H$ and $K$ is closed.

Maybe it's helpful to formulate a somewhat different version of Mikhail's answer, from the viewpoint of the Borel-Tits development of reductive algebraic groups over an arbitrary field. When the field is $\mathbb{R}$, much of the basic structure of Lie groups is recaptured this way including the structure of compact connected Lie groups: such a group is always the group of real points of a connected reductive algebraic group over $\mathbb{C}$. (The book by Onishchik and Vinberg captures much of the information, but at the cost of leaving most proofs as structured exercises and working always over a base field of characteristic 0. The main ideas for algebraic groups work in much more generality.)

In the early part of Borel's book Linear Algebraic Groups you find in I.2.2 a general formulation which implies in particular that the answer to the question asked here is yes. (Indeed, $L$ is connected as well as closed.) The treatment in 7.5 of my book with the same title is a little more explicit, but doesn't refer to fields of definition since those are deferred to the end of the book. As far as I know there is no completely explicit formulation for compact Lie groups in the textbook literature, though I hope I'm wrong.

Unfortunately there seems to be no self-contained textbook treatment of the structure of compact Lie groups, though the representation theory is treated for example in the Springer GTM by Brocker and tom Dieck. Instead, compact Lie groups are typically discussed as special cases of more general Lie groups in larger books.