Extending vector bundles on a given open subscheme
This is true if $X$ satisfies Serre's condition $S_2$, i.e. $\mathcal O_X$ is $S_2$. Then a vector bundle is $S_2$ since locally it is isomorphic to $\mathcal O_X^n$.
More generally, a coherent sheaf $F$ on a Japanese scheme (for example: $X$ is of finite type over a field) which is $S_2$ has a unique extension from an open subset $U$ with $\operatorname{codim} (X\setminus U)\ge 2$. This follows at once from the cohomological characterization of $S_2$.
Thus, another name for the $S_2$-sheaves: they are sheaves which are saturated in codimension 2, and another name for the $S_2$-fication: saturation in codimension 2.
P.S. Of course, by Serre's criterion, normal = $S_2+R_1$. So the above statement is true for any normal (e.g. smooth) variety.
P.P.S. And of course, Gorenstein implies Cohen-Macaulay implies $S_2$. So the statement is also true for hypersurfaces and complete intersections, which could be very singular and non-reduced.
Edit to define some terms:
A Japanese (or Nagata) ring is a ring obtained from a ring finitely generated over a field or $\mathbb Z$ by optionally applying localizations and completions. The property used here is that for a Japanese ring $R$, its integral closure (normalization) $\tilde R$ is a finitely generated $R$-module. This is important because the $S_2$-fication $S_2(R)$ lies between $R$ and $\tilde R$.
A coherent sheaf $F$ satisfies $S_n$ if for any point $x\in Supp(F)$, one has $$ depth_x (F) \ge \min(\dim_x Supp(F),n) $$ If $F$ locally corresponds to an $R$-module $M$, and $x$ to a prime ideal $p$, then the depth is the length of a maximal regular sequence $(f_1,\dots, f_k)$ of elements of $R_p$ for $M_p$ (so, $f_1$ is a nonzerodivisor in $M_p$, etc.).
Let $i:U \to X$ be the embedding. Assume that $i^*F = E$. Then by the adjunction we have a map $F \to i_*E$ which is an embedding, since the kernel is zero on $U$, so it is a torsion sheaf, and a vector bundle doesn't have torsion subsheaves (if $X$ doesn't have embedded components). So, we have an exact triple $0 \to F \to i_*E \to G \to 0$ for some $G$, which is supported on $X \setminus U$. If $X$ satisfies and $codim_X (X\setminus U) \ge 2$, then (provided $S_2$ condition) we have $G^* = {\mathcal Ext}^1(G,O_X) = 0$, so dualizing the sequence we see that $F^* = (i_* E)^\ast$, hence $F = (i_* E)^{\ast\ast}$, so $F$ has to be the reflexive envelope of $i_* E$. This proves the uniquencess. This also allows to construct an example of a vector bundle on $U$ which does not extend to a vector bundle on $X$.
This is false as stated; for example, if $X$ is obtained from a projective geometrically connected smooth surface over a field $k$ by gluing two points together and $U$ is the complement of the singular point, then the kernel of the restriction map from the Picard group of $X$ to the Picard group of $U$ is easily seen to be $k^*$. You need to assume that the depth of all the components of the complement of $U$ in $X$ is at least 2; then the statement is correct. If $j\colon U \to X$ is the embedding, then $j_*j^*F = F$.