How much should the average mathematician know about foundations?
The answer is essentially the same as how much should the average mathematician know about combinatorics? Or group theory? Or algebraic topology? Or any broad area of mathematics... It's good to know some, it's always helpful to know more, but only really need the amount that is relevant to your work. Perhaps a small but significant difference with foundations is that there is a natural curiosity about it, just like I'm naturally curious about the history and geography of where I live, even though I only need minimal knowledge in day-to-day life.
One does need to know where to go when deeper foundational questions arise. So one should keep a logician colleague as a friend, just in case. This entails knowing enough about foundations and logic to engage in casual conversation and, when the need arises, to be able to formulate the right question and understand the answer you get. This is no different than any other area of mathematics.
Some mathematicians may have more than just a casual curiosity about foundations, even if they work in a completely different area. In that case, learn as much as your time and curiosity permits. This is great since, like other areas of mathematics, foundations needs to interact with other areas in order to advance.
So, what do you need to have a casual conversation with a logician? Adjust to personal taste:
- Some understanding of formal languages and the basic interplay between syntax and semantics.
- Some understanding of incompleteness and undecidability.
- Some understanding of the paradoxes that led to the current state of set-theoretic foundations.
- Some understanding that logic and foundations does interact with your discipline.
To address the additional questions regarding sets. Personally, I don't think it's right to say that the notion of set is defined by foundations. It's a perfectly fine mathematical concept though it (sometimes confusingly) has two distinct and equally important flavors.
The main evidence for this point of view is that the notion of set existed well before Cantor and their use was common. Here is one of my favorite early definitions due to Bolzano (Paradoxien des Unendlichen, 1847):
There are wholes which, although they contain the same parts A, B, C, D,..., nevertheless present themselves as different when seen from our point of view or conception (this kind of difference we call 'essential'), e.g. a complete and a broken glass viewed as a drinking vessel. [...] A whole whose basic conception renders the arrangement of its parts a matter of indifference (and whose rearrangement therefore changes nothing essential from our point of view, if only that changes), I call a set.
(See this MO question for additional early occurrences of sets of various shapes and forms.)
What Bolzano describes is the combinatorial flavor of sets: it's a basic container in which to put objects, a container that is so basic and plain that it has no structure of its own to distract us from the objects inside it. There is another flavor to sets in which they are used to classify objects, to put into a whole all objects of the same kind or share a common feature. This usage is also very common and also prior to foundational theories.
Mathematicians use both flavors of sets, often together. For a variety practical reasons, foundational theories tend to focus on one, and formalize standard (albeit awkward) ways to accommodate the other.
So-called "material" set theories (ZFC, NBG, MK) focus on the combinatorial flavor of sets. To accommodate classification, these theories allow for as many collections of objects as possible to be put together in a set (with little to no concern whether this is motivated, necessary, or even useful).
So-called "structural" set theories (ETCS, SEAR, many type theories) focus on the classification flavor of sets. To accommodate combinatorics, these theories include a lot of machinery to relate sets and identify similar objects across set boundaries (with little to no concern about the nature of elements within sets).
Both of these approaches are viable and they both have advantages over the other. However, it's plainly wrong to think that working mathematicians have to choose one over the other, or even worry about the fact that it's difficult to formalize both simultaneously. The fact is that the sets mathematicians use in their day-to-day work are just as suitable as containers as they are as classifiers.
I agree with the previous answer: it is difficult to state what an average mathematician "should" know. (I dare to conjecture that there is no such thing!)
Each university has its own opinion on this: look at the graduate programs, and notice which courses are mandatory. In some countries this is decided by the state.
I can share my own experience of an "average mathematician". As student I has a 1-semester course of logic (it was required), the text was E. Mendelson, Introduction to mathematical logic, but we did not cover the whole book. (In my country the curriculum was established by the state and a course of logic was required). At the same time I read Lyndon, Notes on logic, and I liked this small and very clear book. After my first year, I decided that I will do something else, not logic, and never read anything else on the subject.