When does a system of equations have no solution?

there is no solution when the matrix is $\textbf{inconsistent}$. This means you will have a zero row in your reduced matrix corresponding to a non-zero entry of the desired solution eg.

$$ \left[ \begin{array}{@{}ccc|c@{}} -3&-1&2 & 1 \\ 0& \frac{-5}{3}& \frac{10}{3} & \frac{8}{3} \\ 0&0& 0 & \text{any non-zero} \\ \end{array} \right] $$

this is because the third row would imply $0*x+0*y+0*z = 0 = c \ne 0$ which is obviously false

Given a system of linear equations represented by the matrix equation: $\mathbf{A}\vec{x}=\vec{b}$, there is no unique set of solutions for $\det{\mathbf{A}}=0$.

Therefore, in your case:

$$\begin{bmatrix}-3 & -1 & 2 \\ 0 & -\frac{5}{3} & \frac{10}{3} \\ 0 & 0 & a+2\end{bmatrix}\begin{bmatrix}x \\ y \\ z\end{bmatrix}=\begin{bmatrix}1 \\ \frac{8}{3} \\ b+\frac{6}{5}\end{bmatrix}$$

So we are interested in the case when:

$$\begin{vmatrix}-3 & -1 & 2 \\ 0 & -\frac{5}{3} & \frac{10}{3} \\ 0 & 0 & a+2\end{vmatrix}=0 \implies (5a+10)=0\implies a =-\frac{10}{5}=-2$$

Answering the title of the post: a linear system $$A x = y,\qquad x\in \mathbb R ^n, y\in \mathbb R ^m$$ admits solutions if and only if the matrix $(A\vert y)$, obtained by juxtaposing the column $y$ and the matrix $A$, has the same rank of $A$ itself.

This happens if and only if $y$ is a linear combination of the columns of $A$. Geometrically, this means that $y$ lies in the image of the linear map $x\mapsto Ax$.