What would you call a finite collection of unordered objects that are not necessarily distinct?

If you're looking for something like a set which may have repeated elements, standard terms are multiset or bag. See multiset on wikipedia.

The common term is multiset. For a formal definition, you can for instance define the set of multisets of size $n$ of a given set $A$ as $A^n/\mathfrak{S}_n$ where $\mathfrak{S}_n$ acts by permutation of the factors; or if you don't want to be bothered by size you can define it as a map $f: A\to \mathbb{N}$ where $f(a)$ is supposed to represent the number of times $a$ appears in the multiset.

These are two interesting models for different situations, and there are probably more.

In this context you can identify what you call a $\mathbf{\ set^*}$ with a function that has a finite domain and has $\mathbb N=\{1,2,3\cdots\}$ as codomain.

$A$ and $B$ in your question can both be identified with function: $$\{\langle3,2\rangle,\langle4,2\rangle,\langle8,1\rangle,\langle11,1\rangle\}$$Domain of the function in this case is the set $\{3,4,8,11\}$.