Matrices with rank exactly r as variety
Here is a proof of the irreducibility of the variety $V_r$ of matrices of rank $r$.
Consider the action of $GL(n)\times GL(m)$ on $M(n,m)$ given by $$(G,H)\bullet A=GAH^{-1}$$ Two matrices matrices $A,B\in M(n,m)$ have the same rank if and only if they are in the same orbit.
Hence if one fixes a matrix $A_r$ of rank $r$ , the variety $V_r$ of matrices of rank $r$ is the orbit of $A_r$ under this action , which means that we have a regular surjective morphism $$GL(n)\times GL(m)\twoheadrightarrow V_r:(G,H)\mapsto GA_rH^{-1} $$ Since $GL(n)\times GL(m)$ is irreducible, so is $V_r$.
To find the dimension of our set $V_r$ is not difficult. It is $r(m+n-r)$; cf. for example How to calculate the degrees of freedom of an $r$-ranked matrix with the size being $n\times n$?
EDIT 1. Let $W_r=\{A\mid \det(U_{r+1})=0$ for every $(r+1) \times(r+1) $ submatrix $U_{r+1}$ of $A\}$, be the affine algebraic set of matrices with rank at most $r$. Note that $W_r$ is irreducible in $K^{mn}$ (and $W_{r-1}$ is the singular locus of $W_r$ when $r<n$); consequently, $W_r$ is said to be an affine variety (in fact, the definitions are not really fixed).
$V_r$ is a Zariski open dense subset of $W_r$ and therefore is a quasi-affine variety (or a constructible set or simply a variety).
A Zariski topological space is called irreducible if it cannot be written as a union of two proper closed sets. For this definition, the Zariski open $V_r$ is irreducible. Georges gave a direct proof of this fact.
EDIT 2. Answer to ziggurism. As a simple example, consider $W_{n-1}=\{A=[a_{i,j}]\in M_n;\det(A)=0\}$. If $A$ is a singular point of $W_{n-1}$, then, for every $i,j$,
$\dfrac{\partial \det(A)}{\partial a_{i,j}}=cofactor(a_{i,j})=0$, that is, $A\in W_{n-2}$.