Where is axiom of regularity actually used?
Basic mathematics was done long before set theory. Its users couldn't care less whether or not ZF is the underlying theory of the universe or some other theory. As long as it works fine.
There are a few things to point out:
The natural numbers (standard model of PA), on which classical number theory is developed, are well-ordered regardless to their construction. This is one of their defining properties, after all. Therefore all the inductions we use on them hold regardless to whether or not regularity holds in general.
Classical analysis can be developed, again, without regularity to hold. Most basic theorems (i.e. freshman calculus) are theorems about continuous functions or sequences of real numbers.
Even if the axiom of regularity fails, it still holds in an inner model (granted the other axioms of ZF hold). Therefore even without regularity we can still insist on working with sets for which regularity holds.
The main idea behind having set theory as a foundational theory is that we can carry out the "non set-theoretic" constructions as sets. There is, at least in the basic level of mathematics, little to none interaction between the mathematics and the foundational theories. This is the same reason we don't see proofs assuming CH during our first two-three years of mathematics, these assumptions while interesting are irrelevant for basic mathematics.
Within set theory, the rank function and the Mostowski collapse are already enough reason for assuming it, and these are used a lot.
Here's an (IMO) interesting example. Surely you're familiar with the set-theoretic interpretation of ordered pairs given by $(x,y) = \{ \{ x \}, \{ x, y \} \}$. You may wonder, why not use $(x,y) = \{ x, \{ x, y\}\}$?
We can... if we assume some amount of regularity. If there exists a set $S$ satisfying $S = \{ \{S, T\}, U \}$ with $T \neq U$, then we would have
$$ (S, T) = \{ S, \{ S, T \} \} = \{ \{ \{ S, T \}, U \}, \{ S, T \} \} = ( \{ S, T \}, U ) $$
which contradicts the property of ordered pairs $(S, T) = (X, Y) \implies S = X \wedge T = Y$, however regularity forbids the existence of such a set $S$.