Prove the determinant is $0$
We want to prove that the $(n-1)\times(n-1)$ matrix $$ A_n=\pmatrix{ \dfrac{1}{2!} &\dfrac{1}{3!} & \cdots &\cdots & \cdots &\dfrac{1}{n!} \\ 1 &\dfrac{1}{2!} &\dfrac{1}{3!} &\cdots &\cdots &\dfrac{1}{(n-1)!} \\ 0 & 1 & \ddots & \ddots & \ddots & \vdots\\ \vdots & \ddots & \ddots & \ddots&\ddots & \vdots\\ \vdots & \ddots & \ddots &1& \dfrac{1}{2!} & \dfrac{1}{3!}\\ 0 & \cdots & \cdots & 0 & 1 & \dfrac{1}{2!} } $$ is singular when $n\ge4$ is even. This can be proved by mathematical induction on $n$. The base case $n=4$ can be verified directly. In the inductive step, note that for any positive integer $m$, $$ \sum_{k=0}^m\frac{(-1)^k}{k!}\frac{1}{(m-k)!}=\frac{1}{m!}\sum_{k=0}^m(-1)^k\binom{m}{k}=\frac{1}{m!}(1-1)^m=0. $$ Move the first and the last summands out of the sum, we obtain $$ \frac{1}{m!}+\sum_{k=1}^{m-1}\frac{(-1)^k}{k!}\frac{1}{(m-k)!}=\frac{(-1)^{m+1}}{m!}. $$ Denote the $i$-th row of $A_n$ by $a_i$. The previous identity means that \begin{aligned} u&:=a_1+\sum_{k=1}^{n-1}\frac{(-1)^k}{k!}a_{k+1}\\ &=\left(-\frac{1}{2!},\,\frac{1}{3!},\,-\frac{1}{4!},\,\frac{1}{5!},\,\ldots,\,-\frac{1}{(n-2)!},\,\frac{1}{(n-1)!},\,\frac{1}{(n-1)!}-\frac{1}{n!}\right). \end{aligned} (Note that the last entry of $u$ is not $-\frac{1}{n!}$, because the last column of $A_n$ ends with $\frac{1}{2!}$, not $1$.) In other words, by some appropriate elementary row operations, we can modify the first row of $A$ to $u$. Therefore, by the multilinearity of the determinant function, $\det(A_n)$ remains unchanged if we replace the first row of $A_n$ by $$ v:=\frac12(a_1+u) =\left(0,\,\frac{1}{3!},\,0,\,\frac{1}{5!},\,\ldots,\,0,\,\frac{1}{(n-1)!},\,\frac{1}{2\times(n-1)!}\right). $$ Now suppose $n\ge6$ is even. By Laplace expansion along $v$ and by induction assumption, we get \begin{aligned} \det(A_n) &=-\frac{1}{3!}\det(A_{n-2})-\frac{1}{5!}\det(A_{n-4})-\cdots-\frac{1}{(n-1)!}\det(A_2)+\frac{1}{2\times(n-1)!}\\ &=-\frac{1}{(n-1)!}\det(A_2)+\frac{1}{2\times(n-1)!}\\ &=-\frac{1}{(n-1)!}\frac{1}{2!}+\frac{1}{2\times(n-1)!}\\ &=0. \end{aligned}
Finally I found a direct way to work this out, that is, to use Generating Functions.
From this article we get the following proposition:
Proposition. Consider the following infinite matrix with $1$s in the super diagonal. $$ D=\left[\begin{array}{ccccc} b_{0} & 1 & 0 & 0 & \cdot \\ b_{1} & c_{1} & 1 & 0 & . \\ b_{2} & c_{2} & c_{1} & 1 & . \\ b_{3} & c_{3} & c_{2} & c_{1} & . \\ b_{4} & c_{4} & c_{3} & c_{2} & . \\ \cdot & . & . & . & . \end{array}\right] $$ Let $B(x)=\sum_{n=0}^{\infty} b_{n} x^{n},$ and $C(x)=\sum_{n=1}^{\infty} c_{n} x^{n}$ be the generating functions for the sequences $b_{0}, b_{1}, b_{2}, \ldots$ and $c_{1}, c_{2}, c_{3}, \ldots,$ respectively. If $$ A(x)=\frac{B(x)}{1+C(x)}=\sum_{n=0}^{\infty} a_{n+1} x^{n} $$ then $a_{n}=(-1)^{n-1} D_{n}$ and $1+x A(-x)$ is the generating function of $D_{n}$.
For the original problem, it is equivalent to calculate the determinant: $$ D=\left| \begin{matrix} \dfrac{1}{2!}& 1& 0& \cdots& 0\\ \dfrac{1}{3!}& \dfrac{1}{2!}& 1& \cdots& 0\\ \dfrac{1}{4!}& \dfrac{1}{3!}& \dfrac{1}{2!}& \cdots& 0\\ \vdots& \vdots& \vdots& \ddots& \vdots\\ \dfrac{1}{\left( 2k+2 \right) !}& \dfrac{1}{\left( 2k+1 \right) !}& \dfrac{1}{\left( 2k \right) !}& \cdots& \dfrac{1}{2!}\\ \end{matrix} \right| $$
We can see that $\displaystyle B(x)=\sum _{i=0}^{\infty } \frac{x^{i}}{(i+2)!}= \frac{-x+e^x-1}{x^2}$, $\displaystyle C(x)=\sum _{i=1}^{\infty } \frac{x^i}{(i+1)!} = \frac{-x+e^x-1}{x}$, so $A(x)=\dfrac{1}{x}+\dfrac{1}{1-e^x}$, and finally the generating function for $D_n$ is $D(x)=\dfrac{x}{e^x-1}+x$.
Notice the order $N$ of original matrix $D$ is always odd (since $N=(2k+2)-2+1=2k+3$), so the original claim is equivalent to when $n>4$ and $n$ is odd, $[x^n]D(x)=0$. It's easy to see that $D(x) -1 - x/2$ is an even function, which implies its series only contains terms of the form $x^{2k}$, thus we finished our proof.
I think this method really requires a good understanding of linear equations and Cramer's Law, along with the knowledge of generating functions.