Matrix invertiblity and its Inverse

We can compute the inverse of $M$ with "row-operations" as follows: $$ \left[\begin{array}{cc|cc} A&B&I&0\\0&I&0&I \end{array}\right] \to \\ \left[\begin{array}{cc|cc} I&A^{-1}B&A^{-1}&0\\0&I&0&I \end{array}\right] \to\\ \left[\begin{array}{cc|cc} I&0&A^{-1}&-A^{-1}B\\0&I&0&I \end{array}\right] $$ So that $$M^{-1} = \pmatrix{A^{-1} & -A^{-1}B\\0&I}$$ From there, we can compute $$ (M^TM)^{-1} = M^{-1}(M^T)^{-1} = M^{-1}(M^{-1})^T =\\ \pmatrix{A^{-1} & -A^{-1}B\\0&I} \pmatrix{A^{-1} & -A^{-1}B\\0&I}^T =\\ \pmatrix{A^{-1} & -A^{-1}B\\0&I} \pmatrix{(A^{-1})^T & 0\\-B^T(A^{-1})^T&I}^T = \\ \pmatrix{A^{-1}(A^{-1})^T + A^{-1}BB^T(A^{-1})^T & -A^{-1}B\\ -B^T(A^{-1})^T & I} = \\ \pmatrix{(A^TA)^{-1} + (A^{-1}B)(A^{-1}B)^T & -A^{-1}B\\ -(A^{-1}B)^T & I} $$

Computing the inverse of $ L $ given the inverse of $ M $ is easy: $ L^{-1} = ( M^T M)^{-1} = M^{-1} M^{-T} $.

So, the question really is how to find the inverse of $ M $.

But the inverse of this partitioned triangular matrix is given by $$ \left( \begin{array}{c | c} A & B \\ \hline 0 & I \end{array} \right)^{-1} = \left( \begin{array}{c | c} A^{-1} & -A^{-1} B \\ \hline 0 & I \end{array} \right) $$ (just multiply out.)

So, you can now put the pieces together.