Does there exist a basis for the set of $2\times 2$ matrices such that all basis elements are invertible?

Even without finding such a basis, you can see that singular matrices form a hypersurface in $ M_{n \times n}(F) $ given by the null set of the determinant map $ \det : M_{n \times n}(F) \to F $. When $ F = \mathbb R $, for instance, the set of all singular matrices is a closed subset of $ M_{n \times n}(F) $ (it is the preimage of $ \{ 0 \} $, which is closed, under the continuous determinant map) which is not all of the space, therefore there is an open ball lying outside of this set. As you can prove, we can then find a basis lying in this open ball, hence consisting of invertible matrices.

Explicit counterexamples have been given in other answers, so I will not mention any here.

Consider the counter-example basis: $$\beta:=\left\{\begin{pmatrix} 1&0\\0&1\end{pmatrix},\begin{pmatrix} 0&1\\1&0\end{pmatrix},\begin{pmatrix} 1&0\\1&1\end{pmatrix},\begin{pmatrix} 0&1\\1&1\end{pmatrix}\right\}$$

Let $A=\begin{pmatrix} x_1&x_2\\x_3&x_4\end{pmatrix}$, let the following equation:

$$A=a\begin{pmatrix} 1&0\\0&1\end{pmatrix}+b\begin{pmatrix} 0&1\\1&0\end{pmatrix}+c\begin{pmatrix} 1&0\\1&1\end{pmatrix}+d\begin{pmatrix} 0&1\\1&1\end{pmatrix}$$ Then $$\begin{pmatrix} 1&0&1&0\\0&1&0&1\\0&1&1&1\\1&0&1&1\end{pmatrix}\begin{pmatrix} a\\b\\c\\d\end{pmatrix}=\begin{pmatrix} x_1\\x_2\\x_3\\x_4\end{pmatrix}$$ Check $\det\begin{pmatrix} 1&0&1&0\\0&1&0&1\\0&1&1&1\\1&0&1&1\end{pmatrix}=1$ . So there is an unique solution for $A$, hence $\beta$ is a basis of $2$ by $2$ matrix, where all basis elements are invertible.

\begin{align*} \left[ \begin{matrix} 0 & 1 \\ 1 & 1 \end{matrix} \right], \left[ \begin{matrix} 1 & 0 \\ 1 & 1 \end{matrix} \right], \left[ \begin{matrix} 1 & 1 \\ 0 & 1 \end{matrix} \right], \left[ \begin{matrix} 1 & 1 \\ 1 & 0 \end{matrix} \right] \end{align*}