What exactly is a coset?

If you "multiply some element of $H$ on the left by some element from the group $G$", that is not a coset. If you multiply all elements of $H$ on the left by one element of $G$, the set of products is a coset.

If $H$ happens to be a normal subgroup (i.e. its left cosets are the same as its right cosets), then one can actually multiply cosets, and that gives another group, the quotient group $G/H$.

(I'm having trouble figuring out what you're trying to say in your last paragraph.)

Let's go back to basic set theory. How many elements are in the set:

$$S=\{\{1,2\},\{3,4\}\}$$

If you said $4$, you'd be wrong, there are two - the elements are sets, $\{1,2\}$ and $\{3,4\}$.

While a coset is a set, when we talk about "the set of all cosets," we are actually talking about a set containing sets as members.

You keep talking about individual members of $H$. That is essentially useless. $H$ is a "block" that is acted on uniformly by a single $g$, not individually having different $h\in H$ acted on by different $g\in G$.

A left coset is an equivalence class of $G/\sim$, where $\sim$ is the equivalence relation that states that two elements of the group, $g_1$ and $g_2$, are equivalent if $g_1 = g_2 h$ for some element $h \in H$. This is equivalent to your definition as the set $\{gH : g \in G\}$, since if $g_1 \sim g_2$ we have $g_1 H = g_2 H$, and vice versa. If you can understand why it might be useful to understand equivalence classes, it will become clear why it is interesting to study cosets.

To be more concrete with the computations, suppose that $g_1 \sim g_2$, so that $g_1 = g_2 h$. Then we have $g_1 H = g_2 h H$. Thus, it will suffice to show that $hH = H$. Since $H$ is a subgroup, can you see why multiplying every element on the left by $h \in H$ only shuffles around all the elements, without losing anything? Conversely, if $g_1 H = g_2 H$ as sets, then we know that the left hand side contains an element of the form $g_1 e = g_1$, which must be equal to some element $g_2 h$ on the right hand side.