If the localization of a ring $R$ at every prime ideal is an integral domain, must $R$ be an integral domain?
Zev's and Georges' answers are complete. I would like to give you a deeper sight. This is a guide line through easy claims that you should be able to prove on your own. Let $A$, $B$ be two rings.
Every ideal of $A \times B$ is of the form $I \times J$, for unique ideals $I \subseteq A$ and $J \subseteq B$. If this is the case, $(A \times B) / (I \times J) \simeq (A / I) \times (B / J)$.
$A \times B$ is a domain iff ($A$ is a domain and $B = 0$) or ($B$ is a domain and $A = 0$).
Every prime ideal of $A \times B$ is of the form $\mathfrak{p} \times B$, for some prime ideal $\mathfrak{p}$ of $A$, or $A \times \mathfrak{q}$, for some prime ideal $\mathfrak{q}$ of $B$.
The localization $(A \times B)_{\mathfrak{p} \times B}$ is isomorphic to $A_\mathfrak{p}$, for every prime ideal $\mathfrak{p}$ of $A$.
This proves that if a ring $R$ is the product of a finite number ($\geq 2$) of integral domains, then $R$ is not a domain but every its localization at primes is a domain.
This has also a geometric interpretation, if you know Zariski topology on the prime spectrum of a ring. From (3) you have a homeomorphism $\mathrm{Spec} (A \times B) \simeq \mathrm{Spec} A \coprod \mathrm{Spec} B$, then $\mathrm{Spec} (A \times B)$ is disconnected and is locally the spectrum of a domain, if $A$ and $B$ are domains.
One simple counterexample is $R=\mathbb{Z}/6\mathbb{Z}$. The prime ideals of $R$ are $P=2\mathbb{Z}/6\mathbb{Z}$ and $Q=3\mathbb{Z}/6\mathbb{Z}$ and $R_P$ is an integral domain (the argument easily carries over for $R_Q$), while $R$ clearly is not.
No: $R=\mathbb Q\times \mathbb Q$ .