Does "Science" encompass Mathematics?

This is a good question, but there is no consensus as to a good answer. Some people think mathematics is obviously a science, some people think it obviously isn't, and some just aren't sure.

It's common to include mathematics as a special case of science in general discussions. For example, universities usually classify mathematics under the sciences, and "scientific publishing" would generally be understood to include mathematics. However, there are exceptions, and abstract discussions of science often don't apply very well to mathematics.

If this distinction matters in a given case, then you'll have to discuss it explicitly, since you can never assume everyone will agree by default.

If Math is part of science, what kind of science is it?

The most compelling answer I've heard is that it is a formal science. However, this terminology is somewhat obscure, and it is certainly not a consensus answer to your question.

When people say STEM, do they mean Math is separate from Science?

I don't think the intention is to assert that they are definitely separate, but rather just to have an inclusive acronym we can all agree on.

The first definition of science that Merriam-Webster gives is:

: knowledge about or study of the natural world based on facts learned through experiments and observation

Math doesn't investigate the natural world and thus isn't a science according to that definition.

In How to Think Straight About Psychology Keith E. Stanovich defines 3 main traits of science.

Three of the most important are that (1) science employs methods of systematic empiricism; (2) it aims for knowledge that is publicly verifiable; and (3) it seeks problems that are empirically solvable and that yield testable theories (the subject of the next chapter).

Math isn't employing empiricism and thus doesn't fulfill criteria (1) and (3).

Unfortunately the duplicate in mathematics hasn't got good answers, so I will give it a try.

The answer is: Mu.
It means that the question must be unasked or that neither "yes" nor "no" is right or wrong.

Mathematics is examining the properties of consistent mental models or structures. It started with numbers and geometric figures and it was used for applications (counting, area calculation) from the beginning. It would not be completely wrong to name it "number philosophy" although it has expanded greatly and examines now a innumer...a very great number of concepts.

An example to show the difference to science: Let's say a scientist would try to prove the Pythagorean theoreom without mathematics. He would find out that if we draw squares on the side of a right-angled triangle, the smaller ones look like they have the same area as the big one. He would experiment with it and while they really look very similar, he will never achieve an equal result. The cuts are not completely straight, the material bends, the lines have always some extent. Even if he succeeds in determining that for all tested right triangles the values are mostly equal, he/she can never be sure that it will stay so.

A mathematician can prove that the sides are exactly equal. This is possible because s/he does not use real-world modelling and is therefore not limited by their applications. But while it is not the real-world, it still allows discoveries. It is not self-evident that right triangles have this interesting property.

Mathematics is a necessary part of science because it is a building block for any precise models we need to refine the work. Physical values are models by numbers (or matrices/tensors) and a concept of a dimension. In higher physics countle...a big number of mathematical models are used.

For the reason that mathematicians and other scientists are sharing many mental similarities (curiosity, challenging matters of course and relishing hard, but interesting problems) their faculties are often joined together which results in the STEM field.

So mathematics is "sciency".

But...it is not really a science because it is more fundamental: It does not require knowledge or experience from other scientific fields. If a mathematician travels back in time 20 000 years (the humans were on a comparable intelligence level as today) he would be able to teach a bright kid modern mathematics which is impossible for modern science because there is no infrastructure to replicate experiments. Mathematics also has no room for error or reevaluation. Once the axioms are set (yes, if the axioms change, the result can change), a result is valid for all times. Mathematics is also used massively outside science including the humanities.

So mathematics is not a science.