max and min versus sup and inf
The main difference is that the infimum and supremum always exists (if you allow the values $\pm \infty$) but the minimum and maximum doesn't.
If the set is finite then it is trivial, as a finite set always has a maximal and minimal element, but if you look at something like:
$A = \{x \in \mathbb{Q}: x > 0\}$.
Then you cannot find a smallest element in the set as when you find some candidate for the smallest element, the half of it is still smaller. And $0 \notin A$, therefore the Minimum doesn't exist. In contrast to this the Infimum is just the biggest lower bound and doesn't have to be in $A$ itself, (not even in $\mathbb{Q}$). You can easily see that $\text{inf}(A)=0$.
Another example is $B = \{x \in \mathbb{Q}: x>0,x^2 > 2\}$.
Certainly $\sqrt{2} \notin \mathbb{Q}$ therefore the minimum doesn't exist as for every candidate for your minimum you can go closer to $\sqrt{2}$ and find a smaller one ($\mathbb{Q}$ is dense in $\mathbb{R}$), but the infimum is $\text{inf}(B)=\sqrt{2}$.
Otherwise if the minimum or maximum does exist, it is surely equal to the infimum/supremum as it is the biggest lower / least upper bound in this case.
If we have a set $A$ and some partial ordering on $A$, for $B$ a subset of $A$ then we define the $\sup$ of $B$ (denoted $\sup B$) as the least element $a$ such that:
- $\forall b\in B\colon b<a$ or $b=a$ ($a$ is an upper bound of $B$)
- If $c$ is such that the above condition holds for $c$ (that is $c$ is an upper bound of $B$) then $a<c$ or $a=c$ (namely, every upper bound of $B$ is at least $a$).
In the case that $a=\sup B$ and $a\in B$ we say that $a$ is the maximum of $B$ and denote it by $\max B$. Note that this is not always the case, as presented in other answers $1=\sup \{x\in\mathbb{Q} : x<1\}$ but it is not a maximal element of the set.
The definitions for $\inf$ and $\min$ are symmetric when replacing upper bound by lower bound, etc.
Note, however, that not every order relation has this property of having upper bounds, not even for bounded subsets. It is easy to construct an order in which there is a bounded set which has neither $\sup$ nor $\inf$ (and thus no $\max$ and $\min$ as well).
For example, consider $\{a,b,c\}$ and define $a<b$ and $a<c$ however $b$ and $c$ are incomparable, that is both $b<c$ and $c<b$ are false.
The subset $\{a\}$ is bounded, as $b$ and $c$ are both upper bounds, but neither of those is a least upper bound. In this case $a$ is a minimum element of $\{a\}$, but you can add a few more elements in a similar fashion to have that a subset of two incomparable elements has no $\sup$ nor $\inf$.
When discussing real numbers then we always have a supremum, this is a result of the Dedekind cuts construction of the real numbers, which guarantee that every bounded subset has a supremum and infimum.
For a bounded set $A \subset \mathbb{R}$ we have the following:
- $\max A \ \text{exists and equals} \ \sup A \Longleftrightarrow \sup A \in A$
- $\min A \ \text{exists and equals} \ \inf A \Longleftrightarrow \inf A \in A$