why is that most of the books in mathematics don't include answers?

Several answers to this question come to my mind, some of which are already covered by the comments. Actually, if I'd ever were to write a textbook I'd do the same.

Since you cite Rudin I'd like to note in advance that many consider his books on analysis among the best, and I don't think it is a coincidence that someone writing books which do get that attention and appreciation follows this practice.

Something everyone learning mathematics (not only there) has to learn is the reality (from which you are well protected at school) that you will usually (not sometimes, usually) face problems which you cannot solve. Maybe not at all, at least not with the first attempts or with little effort. You have to find your way to the solution one way or the other, either by thinking harder and smarter, by talking to people, whatever. But you don't get it for free, in real life. Almost never. It is an experience of people who teach (not just math), that this fact has to be taught as well, unless they want to produce many many people leaving university who will then have to learn it too late the hard way.

And, no insult intended, one word of advice: if you find you really need the solutions to make your way through such books, you may want to consider whether you've really chosen the right topic for you.

And one additional remark: why do you need the solution? By working seriously on a problem you may learn more -- even if you cannot solve it -- than by looking up the answer (and, very likely, missing most of the subtleties).

I think it is a commercial decision. Professors prefer it this way.

I personally think having the answers is beneficial. Even if you are confident in your methods, you may have made a dumb mistake (or have a flawed understanding). Usually if you are on top of the course, you can check the answer and if wrong, rework the problem correctly (if needed, checking the book). Also, working problems can be a chore--having immediate feedback is good psychology.

Note that many classic books have the answers in the back (Poly anything, Granville calculus, Hart College Algebra). This used to be much more the norm. I think the change is more of a commercial one and one that emphasizes the primacy of gatekeepers rather than of learning. [I'm sure the professors tell themselves they care about learning and are warding off lazy students, but how many of them really collect and grade daily homework in sufficient quantities to drive needed learning?]