Category theory and set theory: just a different language, or different foundation of mathematics?

I think that Penelope Maddy's article What Do We Want a Foundation to Do? is a good starting point if you want to read some literature. I don't agree with all of Maddy's conclusions but the terminology that she introduces in this article is exceedingly helpful, as well as the very simple but often overlooked point that the concept of a "foundation of mathematics" is a multifaceted one.

Proponents of foundations other than set theory often emphasize what Maddy calls "essential guidance." The argument is that category theory (or whatever) more accurately reflects how mathematicians actually think, or how they actually do mathematics, or what mathematical structures really are. They may be right (although set theory has more resources in this direction than its opponents sometimes acknowledge), but these alternative foundations don't always outdo set theory when it comes to other roles that we might want a foundation to perform. For example, there's "risk assessment"—what axioms do you really need to derive your theorems, and are those axioms "safe"? Or "generous arena"—maybe the proposed alternative foundations are good for homotopy theory but aren't so suitable for the numerical solution of PDEs or the computation of small Ramsey numbers.

Set theory did a remarkable job in the 19th and 20th centuries of unifying mathematics, putting it all on a common foundation, and providing a framework for analyzing questions of consistency and provability. Nowadays it's easy to take that achievement for granted, and assume that all mathematics is "safe" and that if we want to use methods from one branch of mathematics in another then we will always be able to find a way to do so. If one takes that attitude, then "risk assessment" becomes irrelevant and "generous arena" and "shared standard" drop in importance—I can just worry about finding foundations for the kind of mathematics that I care about, and if my foundations are cumbersome for my colleague's kind of mathematics, well, that's my colleague's problem and not mine. On the other hand, if one does still care about generous arena and shared standard and risk assessment, then set theory still has many advantages.

In short, whether to use set theory or category theory as a foundation depends largely on what you want to do. I agree with Harry Gindi that it's best to think of them as playing complementary roles. In particular, for many of the "traditional" roles that people expect from a foundation (e.g., "meta-mathematical corral" is another Maddy term), I don't think set theory has been superseded.

Category theory and set theory are complementary to one another, not in competition. I think this 'debate' is a bit of academic controversialising rather than an actual difference. If you've done a bit of category theory, you will realize how important the category of sets is (for Yoneda's lemma, representability, existence of generators, etc).

Even if you completely buy into homotopy type theory as a foundation for ∞-categories and homotopy theory, the theory of sets reappears in other garb as the theory of 0-types. A theory of sets is too natural an idea to escape.

I just also want to note: If you write out the syntactic version of ETCS, you end up with something that is more or less equivalent to ZFC. The ETCC, on the other hand, is widely considered to be a dead-end.

From the nLab:

As pointed out by J. Isbell in 1967, one of Lawvere’s results (namely, the theorem on the ‘construction of categories by description’ on p.14) was mistaken, which left the axiomatics dangling with insufficient power to construct models for categories. Several ways to overcome these problems where suggested in the following but no system achieved univocal approval (cf. Blanc-Preller(1975), Blanc-Donnadieu(1976), Donnadieu(1975), McLarty(1991)).

As ETCC also lacked the simplicity of ETCS, it rarely played a role in the practice of category theory in the following and was soon eclipsed by topos theory in the attention of the research community that generally preferred to hedge their foundations with appeals to Gödel-Bernays set-theory or Grothendieck universes.

Edit: Just to clarify, I think most mathematicians working in category theory, homotopy theory, algebraic geometry, etc. are more or less agnostic about foundations, as long as they are equivalent in strength to ZFC (or stronger with universes). There have been arguments for ETCS(+Whatever) as a 'better' foundation, but when you get into hairy set-theoretic issues (for example, see the Appendix to lecture 2 of Scholze's notes on condensed mathematics), we are just as likely to work with ZFC because setting up ordinals in ETCS is an added annoyance. I added this edit just to clarify that I am not a partisan of either approach and appreciate both (and am not interested in bringing up this old argument about Tom's paper that I linked!!!)

The best reference I can think of for this is MathOverflow.

Contrary to some of the comments made above, foundational issues are today often a concern in mathematics and computer science. Contrasting foundational schemes is an activity not just limited to researchers in metamathematics or in mathematical logic. It occurs in computer science repeatedly as workers there develop new programming languages, disciplines, and tools for analysis. Mechanical proof checking, program verification, prototyping languages, relation to resource utilization, rapid system development, and other activities benefit from the perspectives offered by one system or another, or by comparing them.

People who frequent this forum often want to understand things more deeply, look for connections or phenomena that may reveal a ubiquitous pattern, or sense of commonality, so that what works for a proof idea in one field can be adapted to other fields. However, the people are raised in different environments, so their perspectives and means of expression vary. It is this variety that is one of the lesser appreciated aspects of MathOverflow: exposure to a wealth of ways of thinking.

Although your questions have been considered before, they are broad enough that I imagine people have only been able to see pieces of the picture, and that the picture is still new enough that data gathering is still going on. If you search MathOverflow (and the Nlab, and perhaps repositories like ArXiv, or proceedings from relevant conferences in computer science as well as mathematics) you will find many of these pieces. For users whose knowledge on this is more extensive than mine, three names pop immediately to mind: Bauer, Blass, Jerabek. (After I get coffee, more names may occur to me.) Looking at some of their answers on this forum may lead you to specific references.

Elsewhere on this forum I have seen a query similar in intent to yours. I answered that really the theories should be considered more as perspectives than as foundational frameworks, because the entirety of mathematics is not captured by just one. These perspectives (or tools) have utility in their variance and possible interplay, not just in their ability to express part of mathematics. But I am unsure that this way of looking at looking helps you in your search.

Gerhard "Speaking As Observer, Not Researcher" Paseman, 2020.05.16.