Huge matrix in my document. Possible ways to make it look better?

According to my belief that no array using \multirow can't be improved by removing it, here's a simpler variation to percusse's answer:

\documentclass{scrartcl}
\usepackage[T1]{fontenc}
%\usepackage[latin1]{inputenc} % not necessary for the example
%\usepackage[english]{babel}
\usepackage{amsmath}

\begin{document}
\[
\left(\begin{array}{@{}c|c@{}}
\\
\mbox{\Huge$I$} & \makebox[12em]{\Huge$0$} \\
\\
\hline
\begin{matrix}
\beta_1 & \alpha \\
\beta_2 & \beta_1 \\
& \beta_2 \\
\beta_3 & & \alpha \\
\beta_4 & & \beta_3 \\
& & \beta_4 \\
\beta_5 & & & \alpha \\
\beta_6 & & & \beta_5 \\
& & & \beta_6 \\
\beta_7 & \alpha \\
\beta_8 & \beta_7 \\
\beta_9 & & \beta_8 \\
\beta_{10} & & \beta_9 \\
\beta_{11} & & & \beta_{10} \\
& & & \beta_{11} \\
\end{matrix} & \mbox{\Huge$I$}
\end{array}\right)
\]
\end{document}

You can remove the identity elements and replace with I.I forgot to put I_{4\times 4} and similarly 15 but you can add it directly.

Still too big but at least it's consistent. I would really go a low rank representation of that 4 column nonzero part and make it a variable.

\documentclass{scrartcl}
\usepackage{amsmath,multirow}
\setcounter{MaxMatrixCols}{19}

\begin{document}
\[\left(
\begin{array}{cccc|ccccccccccccccc}
\multicolumn{4}{c}{\multirow{4}[2]{*}{\Huge $I$}}&%
\multicolumn{15}{c}{\multirow{4}[2]{*}{\Huge$0$}}\\
&&&&&&&&&&&&&&&&&&\\
&&&&&&&&&&&&&&&&&&\\
&&&&&&&&&&&&&&&&&&\\\hline
\beta_1 & \alpha  &         &         &%
\multicolumn{15}{c}{\multirow{15}[2]{*}{\Huge$I$}}\\
\beta_2 & \beta_1 &         &         & & & & & & & & & & & & & & &\\
        & \beta_2 &         &         & & & & & & & & & & & & & & &\\
\beta_3 &         & \alpha  &         & & & & & & & & & & & & & &\\
\beta_4 &         & \beta_3 &         & & & & & & & & & & & & & &\\
        &         & \beta_4 &         & & & & & & & & & & & & & & &\\
\beta_5 &         &         & \alpha  & & & & & & & & & & & & & & &\\
\beta_6 &         &         & \beta_5 & & & & & & & & & & & & & & &\\
        &         &         & \beta_6 & & & & & & & & & & & & & & &\\
\beta_7 & \alpha  &         &         & & & & & & & & & & & & & & &\\
\beta_8 & \beta_7 &         &         & & & & & & & & & & & & & & &\\
\beta_9 &         & \beta_8 &         & & & & & & & & & & & & & & &\\
\beta_{10} &      & \beta_9 &         & & & & & & & & & & & & & & &\\
\beta_{11} &      &         & \beta_{10} & & & & & & & & & & & & & & &\\
        &         &         & \beta_{11} & & & & & & & & & & & & & & & 
\end{array}\right)\]
\end{document}

I'd tempted to write

\[
\left(
  \begin{array}{@{} c | c @{}}
    1_{4} & 0_{4} \\
    \hline
    A &  1_{15}
  \end{array}
\right)
\]

where ...

In general these very large matrices are very hard for the readers to understand and keep track of. If like in this case, it can easily be split into blocks, then that would help readability.