Mathematical expression for $n$ largest values in a set

$B={\rm argmax}_{A'\subset A, |A'|=n}\sum_{a\in A'} a $

I don't think you can do this in any easy way, unless you basically write a sentence to define the set. However, you could do it inductively:

Define $A=A_0$. Write $$B_1:=\{b\in A_0: b\geq a\:\forall\: a\in A_0\}$$ and define $A_1:=A_0\setminus B_1$. Then inductively, write $$B_{i+1}:=\{b\in A_i: b\geq a\:\forall\: a\in A_i\}$$ and define $A_{i+1}:=A_i\setminus B_{i+1}$. Then $B_5$ is the set you are after.

EDIT: In the comments you said that $A$ can be a multiset, and so can have more than one element of a given value. This means that, in the above, $B_5$ could be the entire set $A$. (For example, take $A=\{3, 2, 2, 2, 2, 2, 2, 1, 1, 1, 1, \ldots\}$). So, what you want to do is set $\overline{B}:=B_i$ where $|B_i|\geq 5$ but $|B_{i-1}|<5$, set $C:=B_i\setminus B_{i+1}$ (these are the elements which we want to remove some of) and set $\overline{C}$ to be a subset of $C$ with $|\overline{B}|-5$ elements. Then take $B:=\overline{B}\setminus\overline{C}$.

There has to be a neater way though!