Kähler metric on a Zariski open subset of a non-Kähler manifold

One can consider the following example.

A Moishezon manifold $M$ is a compact connected complex manifold such that the field of meromorphic functions on $M$ has transcendence degree equal to the complex dimension of $M$. Complex algebraic varieties have this property, but the converse is not true if the dimension is at least $3$.

In 1967, Boris Moishezon showed that a Moishezon manifold is a projective algebraic variety if and only if it admits a Kaehler metric. Now take a non-projective (hence non-Kaehler) Moishezon manifold $M$. One can prove that such a manifold admits a bimeromorphic modification $\pi \colon \widetilde{M} \to M$ such that $\widetilde{M}$ is projective.

This implies that there exist an analytic subset $S \subset M$ such that $U:=M - S$ is biholomorphic to an open subset of a projective variety (one can take as $S$ a $1$-codimensional analytic subset).

Hence $U$ admits a Kaehler metric.

For further details see the paper by Shanyu Ji Currents, metrics and Moishezon manifolds, Pacific J. Math. Volume 158, Number 2 (1993), 335-351.

This example is due to BS -- see the edit to the original question.

For an example in dimension two, consider the Hopf surface $X = \left(\mathbb{C}^2\setminus \{0\}\right)/\mathbb{Z}$, where $\mathbb{Z}$ acts as $n\cdot(x,y) = (2^nx,2^ny)$. One can show that $X$ is diffeomorphic to $S^3\times S^1$. It follows that $X$ is non-Kähler because $H^2(X,\mathbb{Z}) = 0$. The $\mathbb{Z}$ action restricts to an action on $\mathbb{C}^*$, which is identified with $\mathbb{C}^*\times \{0\}$, and the smooth curve $E = \mathbb{C}^*/\mathbb{Z}$ of genus 1 includes into $X$ as a submanifold. $Y = X\setminus E$ is then biholomorphic to $E\times \mathbb{C}$ via the map $[(x,y)]\rightarrow \left([y],x/y\right)$. Thus $Y$ admits a Kähler structure. In fact, $Y$ is a quasiprojective variety, as it is isomorphic to an open subset of $E\times \mathbb{P}^1$.

More generally, the Hopf manifolds $X_n = \left(\mathbb{C}^n\setminus \{0\}\right)/\mathbb{Z},\ \ n\geq 2$, are all non-Kähler, as $X_n\simeq S^{2n-1}\times S^1$. The identification $\mathbb{C}^{n-1}\simeq \mathbb{C}^{n-1}\times \{0\}\subset \mathbb{C}^n$ gives rise to an embedding $X_{n-1}\subset X_n$ and, by an argument identical to the one above, we get an isomorphism $Y_n = X_n\setminus X_{n-1} \simeq E\times \mathbb{C}^{n-1}$. Again, $Y_n\subset E\times \mathbb{P}^{n-1}$ is quasiprojective. This gives an example in every dimension $n\geq 2$.