The coarse moduli space of a Deligne-Mumford stack

To start, here are some references. You can look at Conrad's notes as in Adeel's answer; the first page of those notes already contains what you are asking for in some sense.

Other references include the Stacks project, Vistoli's appendix to his landmark paper or the original paper of Deligne and Mumford. Of course, you can also look at the book of Laumon and Moret-Bailly.

Let me now try to explain a bit how to define the coarse moduli space.

Let $X$ be a finite type Deligne-Mumford algebraic stack of finite type over a noetherian scheme $S$ with finite diagonal.

In Remarque 3.19 of LMB (Laumon, Moret-Bailly) the coarse moduli sheaf of $X$ is defined to be the sheafification of the presheaf $$U\to \{ \mathrm{isomorphism \ classes \ of \ objects \ of \ X_U} \}. $$ This sheaf is not an algebraic space in general.

The paper of Keel and Mori proves that $X$ has a coarse moduli space. The coarse moduli space is defined as a morphism $X\to X^c$ of algebraic stacks with $X^c$ an algebraic space satisfying a certain universal property.

As Niels points out in the comments below and as mentioned above, the coarse space of a stack is not necessarily the coarse sheaf.

The representability of the coarse space $X_{coarse}$ of $X$ by a scheme is a difficult problem, as there are many algebraic spaces which are not representable by a scheme. I'm aware of essentially two non-trivial methods to see that $X_{coarse}$ is a scheme. These are GIT (Mumford) and the methods of Viehweg (see his book). It is with these methods that you can show that the coarse moduli space of the moduli stack of

smooth proper curves of genus $g$, or

principally polarized abelian varieties of fixed degree, or

canonically polarized varieties with fixed Hilbert polynomial, or

hypersurfaces of fixed degree $d\geq 3$ in $\mathbb P^n$ with $n\geq 3$, or

polarized K3 surfaces of fixed degree

is a scheme. (In some special cases you can also argue differently.)

Have a look at the following notes:

• Brian Conrad, The Keel-Mori theorem via stacks, pdf.

This is a nice expository account of the Keel-Mori theorem, giving sufficient conditions for an Artin stack to admit a coarse moduli stack.