Is a ring a set?

You only need to see the formality once to never want to see it again.

The ordered pair $(a,b)$ is defined to be, as a set, $\{\{a\},\{a,b\}\}$. So we could say an ordered triple $(a,b,c)$ is an ordered pair $$((a,b),c)=\{\{\{\{a\},\{a,b\}\}\},\{\{\{a\},\{a,b\}\},c\}\}$$ Satisfying ourselves that such a thing is existentially valid we can freely write $(a,b,c)$ to mean the same thing with less clunky notation.

This will make it formal for you. Let $S$ be any set. Then the ring $R$ is any element of the set $$R=(S,f,g)\in \{S\}\times S^{S\times S}\times S^{S\times S},$$ where $f$ and $g$ are functions, elements of $S^{S\times S}$ satisfying: $$f(f(a,b),c)=f(a,f(b,c)),$$ $$f(a,b)=f(b,a),$$ $$\exists e\in S\text{ such that } f(a,e)=a\;\forall a\in S,$$ $$\forall a\in S, \exists b\in S\text{ such that } f(a,b)=e,$$ $$g(g(a,b),c)=g(a,g(b,c)),$$ $$g(a,f(b,c))=f(g(a,b),g(a,c)).$$

Now do yourself a favor, and don't always treat rings with such formality.

I'll try to address both your questions from the viewpoint of category theory.

Is a ring a set?

No.* But we can always treat a ring $R$ as if it were a set, in the following way; there's a functor $$U:\mathbf{Ring} \rightarrow \mathbf{Set}$$ given on objects by $U(S,+,\times) = S$. This is called the underlying set functor (or "forgetful functor to $\mathbf{Set}$") and it allows us to treat rings as if they were sets and morphisms of rings as if they were functions. This in turn allows us to "pull back" structure on $\mathbf{Set}$ to get structure on $\mathbf{Ring}$.

For example, there's a notion of finiteness for sets. Hence we can define that ring $R$ is finite iff the set $U(R)$ is finite. So we've "pulled back" the notion of finiteness across $U$. Similarly, there's a notion of surjectivity for functions. Hence we can define that a ring homomorphism $f : R_0 \rightarrow R_1$ is surjective iff the corresponding underlying function $U(f) : U(R_0) \rightarrow U(R_1)$ is surjective. Again, this pulls surjectivity back across $U$.

*Except in material set theory, in which it is typically assumed that everything is a set, even things like ordered pairs. This doesn't have too much bearing on everyday mathematics, though.

What does "with" mean?

This is a much, much harder question to answer in a satisfactory way; indeed, basic category theory doesn't even attempt to give this question an answer. But if you have some familiarity with double categories, we can indeed give this question an answer; in particular, see Susan Niefield's article here on the gluing construction (but only once you're ready.)