Determine the lie algebra of the subgroup of SO(4)

You are in the following situation. You have a Lie group $H$ (don't want to use the letter $G$ here, since you are using it) and a system of smooth equations $f_1,..., f_n: H\to\Bbb R$ so that $G:=\bigcap_i f^{-1}_i(\{0\})$ is subgroup of $H$. How can you determine the Lie algebra of $G$? Well: $v\in T_1H$ is tangent to $G$ iff the equations $f_i$ remain unchanged when you perturb in direction $v$. This means that $v\in \bigcap_i\ker( d_1f_i)$, and you get another system of equations $d_1f_i(v)=0$ determining the elements Lie algebra.

In your example you have $13$ equations, on the one hand there are the equations $$g_{11}-g_{22}=0, \quad g_{11}-g_{33}=0, \quad g_{11}-g_{44}=0\\ g_{21}+g_{12}=0,\quad g_{21}-g_{43}=0,\quad g_{43}+g_{34}=0\\ g_{31}+g_{42}=0,\quad g_{31}+g_{13}=0, \quad g_{31}-g_{24}=0\\ g_{41}-g_{32}=0,\quad g_{41}+g_{23}=0,\quad g_{41}+g_{14}=0$$ as well as the equation $$g_{11}^2+g_{21}^2+g_{31}^2+g_{41}^2-1=0.$$

If you take the differentials at $1$ of this the first equations remain unchanged, since they are linear, but the last equation becomes: $$2v_{11}\cdot 1+2v_{21}\cdot 0+2v_{31}\cdot 0+2v_{41}\cdot 0=0.$$

Now you can combine these equations with the additional condition that the Lie algebra elements be anti-symmetric (to get elements of $SO(4)$).

It is infinitesimal perturbations from the identity. If $b$, $c$, and $d$ are small, then $a = 1 + O(b^2 + c^2 + d^2)$, so $a$ is constant up to first order. So $$\mathfrak g=\left\{ \begin{pmatrix} 0 & -b & -c &-d\\ b & 0 & -d & c\\ c & d & 0 & -b \\ d &-c & b & 0 \end{pmatrix} : b,c,d\in \mathbb{R} \right\} .$$

How does one find the Lie algebra of a Lie group? Differentiate the map $$(a,b,c,d) \mapsto \begin{pmatrix} a & -b & -c & -d \\ b & a & -d & c \\ c & -d & a & -b \\ d & c & b & a \end{pmatrix}\quad \mbox{and the equation}\quad a^2+b^2+c^2+d^2=1$$at $(1,0,0,0)$ and evaluate at $(x,y,z,w)$. So $$\mathfrak{g} = \left\{\begin{pmatrix} x & -y & -z & -w \\ y & x & -w & z \\ z & -w & x & y \\ w & z & y & x \end{pmatrix} \mid x,y,z,w \in \Bbb R \mbox{ and }2x+0y+0z+0w = 0 \right\},$$which of course agrees with Stephen's answer $$\mathfrak{g} = \left\{\begin{pmatrix} 0 & -y & -z & -w \\ y & 0 & -w & z \\ z & -w & 0 & y \\ w & z & y & 0 \end{pmatrix} \mid y,z,w \in \Bbb R \right\}.$$It is the general principle that to find the equation of a tangent space to a submanifold, you differentiate the equation defining it. Also, $G \cong \Bbb S^3$ is isomorphic to the group of unit quaternions, as the general expression for an element of $G$ is in the image of the composition of the maps $$\Bbb H \ni z+wj \mapsto \begin{pmatrix} z & -\overline{w} \\ w & z\end{pmatrix} \in \mathfrak{gl}(2,\Bbb C)\quad\mbox{and}\quad \Bbb C \ni a+bi \mapsto \begin{pmatrix} a & -b \\ b & a\end{pmatrix} \in \mathfrak{gl}(2, \Bbb R).$$