Limit of decomposable bundles

What ulrich writes is correct. I will delete this answer if ulrich posts his answer. For me, the fastest way to think about this is via smoothness and irreducibility of the stack of locally free sheaves of specified rank and degree on a curve $X$.

For a fixed smooth, projective curve $X$ over a field $k$, there is an algebraic stack $\text{Bun}_{r,d}(X)$ parameterizing locally free $\mathcal{O}_X$-modules of rank $r$ and total degree $d$. This stack is smooth, since the obstruction group $\text{Ext}^2_{\mathcal{O}_X}(E,E)$ is zero. The stack is not quasi-compact (the slopes in the Harder-Narasimhan filtration of $E$ can be arbitrary integers subject to the constraint coming from $r$ and $d$). However, every quasi-compact open substack $V$ is integral in the sense that there is a representable, fppf (even smooth) $1$-morphism $p:V_0\to V$ from a (geometrically) integral $k$-scheme $V_0$. This follows, for instance, by writing every bounded family of locally free sheaves $E$ as quotients of some fixed locally free sheaf $\mathcal{O}(-N\cdot A)^{\oplus (r+1)}$ where $A$ is a fixed divisor class on $X$ of positive degree, and $N\gg 0$ is a sufficiently positive integer. In that case, $V_0$ is an open subset of an affine space bundle over the Jacobian of $X$.

For $X$ an elliptic curve, there is an open substack $U\subset \text{Bun}_{2,0}(X)$ parameterizing locally free sheaves of the form $E=L\oplus M$ where both $L$ and $M$ are invertible sheaves of degree $0$ such that $L\not\cong M$, e.g., $L\oplus \mathcal{O}_X$ where $L\not \cong \mathcal{O}_X$. The key to openness of this substack is the fact that every infinitesimal deformation of $L\oplus M$ is again of the form $L'\oplus M'$, since $$\text{Ext}^1_{\mathcal{O}_X}(L,L)\oplus \text{Ext}^1_{\mathcal{O}_X}(M,M)\to \text{Ext}^1_{\mathcal{O}_X}(L\oplus M, L\oplus M)$$ is surjective. This uses the fact that $\text{Ext}^1_{\mathcal{O}_X}(L,M)$ and $\text{Ext}^1_{\mathcal{O}_X}(M,L)$ are zero on a genus $1$ curve for invertible $\mathcal{O}_X$-modules $L$, $M$ of degree $0$ such that $L\not\cong M$.

Because of integrality of $V_0$, the open subset $U$ is "dense" in $\text{Bun}_{2,0}(X)$: the inverse image of $U$ in $V_0$ is a nonempty open in the integral scheme $V_0$. Since the nontrivial extension of $\mathcal{O}_X$ by $\mathcal{O}_X$ gives a geometric point of the stack, for any geometric point of $V_0$ mapping to that point, there is a smooth affine curve in $V_0$ that contains that point and whose intersection with the inverse image of $U$ is dense.

Edit addressing higher genus. For a smooth, projective $k$-curve $Y$ of genus $g\geq 1$ such that there exists a flat $k$-morphism $f:Y\to X$, the pullback by $f$ of the genus $1$ example gives an example on $Y$.

Though Jason Starr has given an excellent answer above, let me just describe another path which might be of interest.

Constructing deformations as required by abx on the projective space $P$ is equivalent to finding a vector bundle $G$ on $P$ with a nilpotent endomorphism $t$ and a map $\phi:E\to G$ such that $\phi(E)+t(G)=G$. Then you can write down a family on $P\times A$ where $A=\mathbb{A}^1=\mathrm{Spec} k[t]$ (same $t$ intentionally) by treating $G$ as a sheaf over $X\times A$ supported on $t^n=0$ for some $n$. Then, you have a surjection on $P\times A$, $E\to G$, using $\phi$ and the kernel is a vector bundle whose generic fibers over $A$ are $E$. In general, if $G$ is not a direct sum of line bundles and $E$ is, you get a family whose general member is decomposable and special member is not.

On $P=\mathbb{P}^2$, this is easy to do and for higher dimensional case (at least in characteristic zero) this is an open question. Here is one of the simplest such. Take the usual resolution of the ideal sheaf $I$ of a point and take the push-out $\mathcal{O}(-2)\to \mathcal{O}$, by a quadric not passing through the point to get $0\to \mathcal{O}\to G\to I\to 0$, a rank two vector bundle with a nilpotent endomorphism, got by sending $G\to I\to \mathcal{O}\to G$ and $\phi:\mathcal{O}(-1)^2\to G$.

This bundle gives you a family of rank two vector bundles on $P$ with general member $\mathcal{O}(-1)^2$ and special member indecomposable and thus unstable.

Finally, if $M$ is a rank 2 unstable non-split bundle on $P$, one may assume after twisting, that we have an exact sequence $0\to \mathcal{O}(k)\to M\to J\to 0$, $J$ defining a zero dimensional non-empty scheme. If we restrict this to a curve not passing through $V(J)$, $M$ splits if and only if it splits as $\mathcal{O}(k)\oplus\mathcal{O}$. If we take the curve $C$ to be of sufficiently large degree, one can make sure the map $H^0(M)\to H^0(M|C)$ to be an isomorphism and then $M|C$ can not be split.