Why probability of picking a random prime is 0?
The sentence "The probability of picking a number at random and having it be prime is zero." is, mathematically speaking, either very sloppy or plain wrong (depending on who you ask) and, if you ask me, is a clear demonstration of why there's always a bit of tension between mathematicians and physicists. We call them sloppy, they call us hair splitters.
The correct sentence would be this:
If $p_n$ is the probability of picking a prime when (uniformly) selecting a random number from $1$ to $n$, then $\lim_{n\to\infty} p_n = 0$.
This statement follows directly from the prime number theorem. That theorem tells us that the if $P_n$ is the number of primes smaller than or equal to $n$, then $$\lim_{n\to\infty}\frac{P_n}{\frac{n}{\log n}} = 1.$$ Clearly, we have $p_n=\frac{P_n}{n}$, which means that $$\lim_{n\to\infty} p_n=\lim_{n\to\infty}\frac{P_n}{n} = \lim_{n\to\infty}\left(\frac{P_n}{\frac{n}{\log n}}\cdot\frac{1}{\log n}\right) = \lim_{n\to\infty}\frac{P_n}{\frac{n}{\log n}} \cdot\lim_{n\to\infty}\frac{1}{\log n} = 1\cdot 0=0$$
This also tells you that $p_n\approx \frac{1}{\log n}$ for large values of $n$, so you also know the speed at which $p_n$ converges to $0$ (rather slowly, in fact).
(*) The statement is wrong or sloppy because of a simple reason: there is a lot left out in the statement "pick a random number". What's the distribution? Uniform? There is no uniform distribution over all integers! OK, which distribution are we talking about then? Because surely, there exist probability distributions over $\mathbb N$ with a nonzero probability of picking a random number. For example, picking a random number by throwing a 6 sided die has a $0.5$ chance of picking a prime number.
From the Prime number theorem one can interpret as saying that if you pick a random integer of size about n, then the probability that it is prime is about $\frac{1}{\ln n}$.
If for one second, you relax the size about n condition, then the answer depends on what you mean by "picking an integer randomly."
The problem is that there is no way to pick an integer uniformly at random, which precisely means that if every integer has the same probability of being picked then that probability has to be zero, there is no probability distribution on the positive integers that assigns equal weight to every integer.
If you were to let $S\subseteq \mathbb{N}$ be a set of positive integers, and for every positive integer $n$, let $S_n$ be the set of all $k\in S$ such that $k\le n$. Let $|S_n|$ be the number of elements in $S_n$. Then $$\lim_{n\to\infty} \frac{|S_n|}{n},$$ if it exists, can be viewed as a measure of how large $S$ is. The measure for how large the primes is $0$.
The probability of picking a random positive integer and is prime is defined as $$ \lim_{n\to\infty}\frac{1}{n}|\{k : 1\le k\le n\,\,\,\&\,\,\,\text{$k$ prime}\}| $$ where $|A|$ is the number of elements of the set $A$ (i.e., its cardinal number.)
Prime Number Theorem implies that the above limit is equal to zero.