Why do the real numbers have the least upper bound property while the rationals do not?
The proof that $\mathbb{R}$ does indeed have the Least Upper Bound property really depends upon how you're defining the real numbers; for example, if $\mathbb{R}$ is constructed using Dedekind cuts, then the proof is rather straight-forward and easy. If you're constructing $\mathbb{R}$ using equivalence classes of Cauchy sequences, then it's involved (at least the proofs I've seen/done).
Thus, I'll use this answer to address the concerns of why the Least Upper Bound property needs to be mentioned at all, and why it doesn't trivially hold true for everything.
Firstly, the Least Upper Bound property is essentially the reason calculus can be done; as we shall see, there are "gaps" in the rational numbers. The ability to take limits, which is central to everything done in Real Analysis, is closely related to the Least Upper Bound property. To show why this is the case, I'll quote some equivalences:
- Least Upper Bound Property
- Bolzano-Weierstrass Theorem (all cauchy sequences are convergent) and the Archimedean property
- Monotone Convergence Theorem and the Archimedean property
- Nested Intervals Theorem
All of the above are equivalent, and all are central to Real Analysis.
As far as the Least Upper Bound property not holding more generally in, for example, $\mathbb{Q}$, consider the following set: $$\{ q\in \mathbb{Q} \mid q>0 \wedge q^2<2 \}$$ It is clearly bounded above; 2 is an upper bound, for example. Does this set have a least upper bound? In $\mathbb{R}$ it certainly would, and would be $\sqrt{2}$; since $\mathbb{Q}$ is dense in $\mathbb{R}$, if there were a least upper bound in $\mathbb{Q}$ for this set, then it would also be a least upper bound of the set in $\mathbb{R}$. But we know that $\sqrt{2}$ is irrational, so this cannot be the case. Thus, $\mathbb{Q}$ cannot have the Least Upper Bound property. This is why the Real numbers are necessary.
Take any $x_\infty\in\Bbb R\setminus \Bbb Q$.
You can find an increasing sequence of rational numbers $\left(x_n\right)_{n\in \Bbb N}\in\Bbb Q^\Bbb N$ so that $\lim\limits_{n\to+\infty}x_n=x_\infty$. For example, taking the decimal expansion and truncating after the $n^{th}$ digit would work.
Then, take $\left\{x_n\mid x\in \Bbb N\right\}$. This set is a subset of $\Bbb Q$ but it has no upper bound: If you suppose that $x_k$ is an upper bound, you get a contradiction because $x_k<x_{k+1}$.
That's the problem of rational number: you can have sequences of them that converge to something outside of the rational numbers. But since we can't really talk about the limit if we restrict ourselves to $\Bbb Q$, we have to define a new notion: Cauchy sequences.
$u_n$ is a Cauchy sequence iff $\forall \varepsilon>0, \exists N\in \Bbb N, \forall n \ge N, \forall m\ge N, \left|u_n-u_m\right|<\varepsilon$
The idea is that if you give me a small $\varepsilon$, I can give you a $N\in \Bbb N$ so that if you draw a circle of radius $\varepsilon$ around $u_n$ for $n\ge N$, it'll contain all the $u_m$ for $m\ge N$.
And that's what real numbers are: Cauchy sequences. You have a circle that you can make as small as you want by giving smaller and smaller $\varepsilon$s and you imagine a point it converges to even though that point is not in $\Bbb Q$.
The idea is that simple. Then you have to define the operations on Cauchy sequences. For example $(u+v)_n=u_n+v_n$. But then you have to prove that if $u$ and $v$ are Cauchy sequences (think "real number"), then $u+v$ is a Cauchy sequence (thing "real number"). There are a few additionnal problems like the fact that there are several Cauchy sequences that converge to the same point. For example if you just change the $u_0$. So a real number is the set of all Cauchy sequences that converge towards the point we imagined to represent that real number. But then having sets makes defining $+$ harder. Etc. It's long and fastidious but in the end, you get all the properties of the real numbers that you know.
Anyway, if you had to remember something from this, it would be that there are sequences of rationnal number that look like they converge to some non-rational number so we create those numbers and represent them by sequences converging towards them.
To link this to your question about upper bounds, note that either the upper bound is in the set or there is a sequence of elements of the set that converges to your upper bound.