Fractions seem too large in a column vector

A TABstack allows the inter-row baselineskip to be specified, while still keeping the baselineskip uniform, which I find important for vector/matrix notation.

\documentclass{article}
\usepackage{tabstackengine}
\begin{document}
\[
\setstackgap{L}{1.3\baselineskip}
\mathbf{X} = \bracketVectorstack{ -4 \\ - \frac{5}{2} \\ \frac{9}{2}}
\]
\end{document}

You can obtain a correct vertical padding with the cellspace package, which lets you define minimal vertical paddings at top and bottom of cells in column with specifier prefixed with the letter S (or C if you load siunitx). For matrices, which have no specifier, this is done with the [math] option of the package.

If you want a left or right alignment of the elements in a matrix, you can use the xmatrix* environments defined by mathtools. However, cellspace does not work with these environments . So, either you use a xmatrix environment (centred) and the classic trick of \phantom{...} elements where necessary, or you use the \setcellgapes{xxx}\makegapedcells construct defined by makecell.

Last point: I used the medium-sized fractions from nccmath, as I don't the size discrepancy between display style numbers and textstyle fractions.

Here is a demo:

\documentclass{article}
\usepackage{mathtools, nccmath}
\usepackage[math]{cellspace}
\setlength{\cellspacetoplimit}{2pt}
\setlength{\cellspacebottomlimit}{2pt}
\usepackage{makecell}
\setcellgapes{2pt}

\begin{document}

\[
\begin{bmatrix}-4 \\ - \mfrac{5}{2} \\ \phantom{-}\mfrac{9}{2} \end{bmatrix} \qquad
\makegapedcells\begin{bmatrix*}[r] -4 \\ - \mfrac{5}{2} \\ \mfrac{9}{2} \end{bmatrix*}
\]

\end{document} 

I would like to suggest that you load the mathtools package. In addition to automatically loading the amsmath package, the mathtools package provides "starred" variants of the pmatrix, bmatrix, and vmatrix environments; these permit either left- or right-alignment of the cell contents. (The non-starred variants of the matrix-like environments automatically center the cell contents horizontally.)

\documentclass{article}
\usepackage{mathtools} % for 'bmatrix*' env.
\begin{document}
\[
\begin{bmatrix*}[r] 
   -4 \\[0.5ex]  -\frac{5}{2} \\[1ex]  \frac{9}{2} 
\end{bmatrix*}
\]
\end{document}